[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:43 dist-test-slave-5k54 krb5kdc[10003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:43 dist-test-slave-5k54 krb5kdc[10003](info): set up 2 sockets
Aug 07 22:49:43 dist-test-slave-5k54 krb5kdc[10003](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 22:49:44.950516 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:49:45 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:49:45.146020 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44521
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:45.498124 10016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:45.498492 10016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:45.498598 10016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:45.505874 10016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:45.505985 10016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:45.506040 10016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:45.506099 10016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:45.506145 10016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:45.516506 10016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:45.518312 10016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:45.520162 10016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:45.537987 10026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:45.538146 10022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:45.537981 10023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:45.541169 10016 server_base.cc:1034] running on GCE node
I20260807 22:49:45.542474 10016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:45.544776 10016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:45.546025 10016 hybrid_clock.cc:648] HybridClock initialized: now 1786142985545955 us; error 91 us; skew 500 ppm
I20260807 22:49:45.546527 10016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:45 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142985, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:45.550297 10016 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:49:45.552712 10016 webserver.cc:460] Webserver started at http://127.9.194.254:42583/ using document root <none> and password file <none>
I20260807 22:49:45.553654 10016 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:45.553802 10016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:45.554128 10016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:45.558117 10016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "d3c7873d6f3a45628b60003477304c2b"
format_stamp: "Formatted at 2026-08-07 22:49:45 on dist-test-slave-5k54"
server_key: "96d7be13ceaf06f9d8513e85cbb71978"
server_key_iv: "5c4905dd713e59ce059820d704ca0f32"
server_key_version: "encryptionkey@0"
I20260807 22:49:45.558866 10016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "d3c7873d6f3a45628b60003477304c2b"
format_stamp: "Formatted at 2026-08-07 22:49:45 on dist-test-slave-5k54"
server_key: "96d7be13ceaf06f9d8513e85cbb71978"
server_key_iv: "5c4905dd713e59ce059820d704ca0f32"
server_key_version: "encryptionkey@0"
I20260807 22:49:45.564904 10016 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 22:49:45.568902 10034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:45.570520 10016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:49:45.570832 10016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d3c7873d6f3a45628b60003477304c2b"
format_stamp: "Formatted at 2026-08-07 22:49:45 on dist-test-slave-5k54"
server_key: "96d7be13ceaf06f9d8513e85cbb71978"
server_key_iv: "5c4905dd713e59ce059820d704ca0f32"
server_key_version: "encryptionkey@0"
I20260807 22:49:45.571043 10016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:45.608356 10016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:45.613641 10016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:45.614073 10016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:45.636854 10016 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:44521
I20260807 22:49:45.636868 10089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:44521 every 8 connection(s)
I20260807 22:49:45.639158 10016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:45.644471 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10016
I20260807 22:49:45.644989 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:49:45.645516 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:45.647836 9995 external_mini_cluster.cc:1428] Setting key bcfd9439e4852cd3f27b14afe19d3352
I20260807 22:49:45.661128 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b: Bootstrap starting.
I20260807 22:49:45.664525 10090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:45.666172 10090 log.cc:824] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:45.670639 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b: No bootstrap required, opened a new log
I20260807 22:49:45.680778 10090 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } }
I20260807 22:49:45.681248 10090 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:45.681456 10090 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c7873d6f3a45628b60003477304c2b, State: Initialized, Role: FOLLOWER
Aug 07 22:49:45 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:45.682673 10090 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } }
I20260807 22:49:45.683144 10090 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:45.683518 10090 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:45.683809 10090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:45.686765 10090 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } }
I20260807 22:49:45.687661 10090 leader_election.cc:302] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3c7873d6f3a45628b60003477304c2b; no voters:
I20260807 22:49:45.688576 10090 leader_election.cc:288] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:45.688840 10095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:45.692277 10090 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:45.696026 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:45.654945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.9.194.254:44521)
0807 22:49:45.656221 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:45.656240 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:45.657971 (+ 1731us) server_negotiation.cc:407] Connection header received
0807 22:49:45.668037 (+ 10066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:45.668061 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:45.668765 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:45.669359 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:45.672463 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:45.674664 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:45.676698 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:45.677781 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:45.678045 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:45.684823 (+ 6778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:45.684917 (+ 94us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:45.684935 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:45.684975 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:45.688322 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:45.689399 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:45.689414 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:45.689421 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:45.689698 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:45.690177 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:45.690183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:45.690186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:45.691043 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:45.691730 (+ 687us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:45.692996 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0807 22:49:45.693432 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":449,"threads_started":1}
I20260807 22:49:45.696043 10095 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [term 1 LEADER]: Becoming Leader. State: Replica: d3c7873d6f3a45628b60003477304c2b, State: Running, Role: LEADER
I20260807 22:49:45.699496 10095 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } }
I20260807 22:49:45.705180 10096 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3c7873d6f3a45628b60003477304c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } } }
I20260807 22:49:45.705696 10096 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:45.706410 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c7873d6f3a45628b60003477304c2b. Latest consensus state: current_term: 1 leader_uuid: "d3c7873d6f3a45628b60003477304c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c7873d6f3a45628b60003477304c2b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44521 } } }
I20260807 22:49:45.706743 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:45.709880 10108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:45.716246 10108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:45.729207 10108 catalog_manager.cc:1347] Generated new cluster ID: b604fa1ccf894549a4ec96829781def3
I20260807 22:49:45.729324 10108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:45.760650 10108 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:45.762104 10108 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:45.776161 10108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3c7873d6f3a45628b60003477304c2b: Generated new TSK 0
I20260807 22:49:45.777446 10108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:49:45.824007 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:46.166970 10118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:46.167230 10118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:46.167326 10118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:46.174260 10118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:46.174355 10118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:46.174481 10118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:49:46.183908 10118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:46.185088 10118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:46.186329 10118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:46.200204 10123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:46.200232 10124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:46.200426 10126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:46.201380 10118 server_base.cc:1034] running on GCE node
I20260807 22:49:46.201987 10118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:46.202989 10118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:46.204221 10118 hybrid_clock.cc:648] HybridClock initialized: now 1786142986204171 us; error 42 us; skew 500 ppm
I20260807 22:49:46.204548 10118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:46 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142986, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:46.207831 10118 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:49:46.209443 10118 webserver.cc:460] Webserver started at http://127.9.194.193:35941/ using document root <none> and password file <none>
I20260807 22:49:46.210126 10118 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:46.210278 10118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:46.210608 10118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:46.213748 10118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "6731d9e2cc7745ba96e147cbbc07cf1e"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "1b775f55cf2fb9827be933d25de27d5a"
server_key_iv: "426a3803d639f4ec20d13399feb04ab5"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.214540 10118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "6731d9e2cc7745ba96e147cbbc07cf1e"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "1b775f55cf2fb9827be933d25de27d5a"
server_key_iv: "426a3803d639f4ec20d13399feb04ab5"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.219707 10118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 22:49:46.223279 10134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.224646 10118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:46.224835 10118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "6731d9e2cc7745ba96e147cbbc07cf1e"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "1b775f55cf2fb9827be933d25de27d5a"
server_key_iv: "426a3803d639f4ec20d13399feb04ab5"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.225033 10118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:46.237469 10118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:46.241413 10118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:46.241750 10118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:46.242820 10118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:46.244549 10118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:46.244653 10118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.244868 10118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:46.244951 10118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.279800 10118 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:33847
I20260807 22:49:46.279846 10255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:33847 every 8 connection(s)
I20260807 22:49:46.281088 10118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:49:46.282624 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10118
I20260807 22:49:46.282851 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:49:46.283540 9995 external_mini_cluster.cc:1428] Setting key 315d757fe50593a851c319f877c85770
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Aug 07 22:49:46 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142986, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:46.314671 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:46.286637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:39763 (local address 127.9.194.254:44521)
0807 22:49:46.294200 (+ 7563us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:46.294204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:46.294223 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:49:46.296460 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:46.296464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:46.296536 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:46.296628 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:46.298776 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.299638 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.302278 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.302680 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.302786 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:46.308677 (+ 5891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:46.308683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:46.308686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:46.308718 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:46.311571 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:46.312256 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:46.312263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:46.312268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:46.312435 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:46.312935 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:46.312938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:46.312940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:46.313188 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:46.313297 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:46.314241 (+ 944us) server_negotiation.cc:299] Negotiation successful
0807 22:49:46.314417 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7464,"thread_start_us":7364,"threads_started":1}
I20260807 22:49:46.316284 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:46.287496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44521 (local address 127.9.194.193:39763)
0807 22:49:46.293264 (+ 5768us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:46.293322 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:46.296168 (+ 2846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:46.296897 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:46.296910 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:46.297774 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:46.298605 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:46.298632 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.299785 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.299792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:46.302138 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:46.302154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.302927 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.302935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:46.303271 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:46.304092 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:46.304128 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:46.308451 (+ 4323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:46.311705 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:46.311711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:46.311724 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:46.312131 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:46.312565 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:46.312572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:46.312579 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:46.312820 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:46.313324 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:46.313335 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:46.313765 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 22:49:46.314164 (+ 399us) client_negotiation.cc:240] Negotiation successful
0807 22:49:46.314743 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5262,"thread_start_us":4999,"threads_started":1}
I20260807 22:49:46.318202 10256 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44521
I20260807 22:49:46.318693 10256 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:46.319861 10256 heartbeater.cc:511] Master 127.9.194.254:44521 requested a full tablet report, sending...
I20260807 22:49:46.324589 10052 ts_manager.cc:194] Registered new tserver with Master: 6731d9e2cc7745ba96e147cbbc07cf1e (127.9.194.193:33847)
I20260807 22:49:46.326807 10052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:39763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:49:46.335301 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:46.662014 10262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:46.662276 10262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:46.662377 10262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:46.669713 10262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:46.669829 10262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:46.669948 10262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:49:46.680500 10262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:46.681735 10262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:46.683125 10262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:46.697268 10271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:46.697345 10268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:46.697341 10269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:46.700969 10262 server_base.cc:1034] running on GCE node
I20260807 22:49:46.701619 10262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:46.702580 10262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:46.703769 10262 hybrid_clock.cc:648] HybridClock initialized: now 1786142986703729 us; error 31 us; skew 500 ppm
I20260807 22:49:46.704108 10262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:46 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142986, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:46.707358 10262 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:49:46.708989 10262 webserver.cc:460] Webserver started at http://127.9.194.194:46281/ using document root <none> and password file <none>
I20260807 22:49:46.709640 10262 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:46.709735 10262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:46.710006 10262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:46.712984 10262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "95988394cc8443cf8d62a30fcaa21274"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "471a844f1dd572aaa6a8158d234f91f4"
server_key_iv: "9c9e6182cb9e2e1c725c653bbe597f1c"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.713692 10262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "95988394cc8443cf8d62a30fcaa21274"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "471a844f1dd572aaa6a8158d234f91f4"
server_key_iv: "9c9e6182cb9e2e1c725c653bbe597f1c"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.718773 10262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 22:49:46.722195 10278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.723611 10262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:46.723774 10262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "95988394cc8443cf8d62a30fcaa21274"
format_stamp: "Formatted at 2026-08-07 22:49:46 on dist-test-slave-5k54"
server_key: "471a844f1dd572aaa6a8158d234f91f4"
server_key_iv: "9c9e6182cb9e2e1c725c653bbe597f1c"
server_key_version: "encryptionkey@0"
I20260807 22:49:46.723990 10262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:46.753250 10262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:46.757294 10262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:46.757589 10262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:46.758548 10262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:46.759934 10262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:46.760046 10262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.760195 10262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:46.760295 10262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:46.795367 10262 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:38245
I20260807 22:49:46.795406 10400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:38245 every 8 connection(s)
I20260807 22:49:46.796712 10262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:49:46.802830 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10262
I20260807 22:49:46.803040 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:49:46.803627 9995 external_mini_cluster.cc:1428] Setting key 6d30ae6537ff58808c823fa70965bbde
Aug 07 22:49:46 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142986, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:46.819520 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:46.799301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:36109 (local address 127.9.194.254:44521)
0807 22:49:46.799522 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:46.799529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:46.800874 (+ 1345us) server_negotiation.cc:407] Connection header received
0807 22:49:46.802884 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:46.802888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:46.803000 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:46.803117 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:46.805123 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.805906 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.808448 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.808863 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.808971 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:46.813701 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:46.813706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:46.813709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:46.813741 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:46.816400 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:46.817155 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:46.817167 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:46.817169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:46.817299 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:46.817754 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:46.817759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:46.817761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:46.818044 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:46.818203 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:46.818884 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 22:49:46.819131 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:49:46.820508 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:46.799921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44521 (local address 127.9.194.194:36109)
0807 22:49:46.800721 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:46.800764 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:46.802578 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:46.803387 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:46.803399 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:46.804170 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:46.804960 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:46.804979 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.806074 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.806080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:46.808308 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:46.808326 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:46.809092 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:46.809101 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:46.809431 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:46.810165 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:46.810198 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:46.813466 (+ 3268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:46.816552 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:46.816562 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:46.816580 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:46.817029 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:46.817413 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:46.817418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:46.817422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:46.817627 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:46.818174 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:46.818182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:46.818533 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 22:49:46.818794 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 22:49:46.819327 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":364,"threads_started":1}
I20260807 22:49:46.822050 10401 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44521
I20260807 22:49:46.822405 10401 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:49:46.823285 10401 heartbeater.cc:511] Master 127.9.194.254:44521 requested a full tablet report, sending...
I20260807 22:49:46.824844 10052 ts_manager.cc:194] Registered new tserver with Master: 95988394cc8443cf8d62a30fcaa21274 (127.9.194.194:38245)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
I20260807 22:49:46.825946 10052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:36109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:49:46.851074 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:47.188491 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:47.188786 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:47.188894 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:47.196782 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:47.196907 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:47.197023 10406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:49:47.207237 10406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:47.208606 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:47.209957 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:47.223654 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:47.223855 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:47.224485 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:47.224609 10406 server_base.cc:1034] running on GCE node
I20260807 22:49:47.225246 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:47.226222 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:47.227543 10406 hybrid_clock.cc:648] HybridClock initialized: now 1786142987227473 us; error 49 us; skew 500 ppm
I20260807 22:49:47.227967 10406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:47.231519 10406 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:49:47.233053 10406 webserver.cc:460] Webserver started at http://127.9.194.195:35695/ using document root <none> and password file <none>
I20260807 22:49:47.233774 10406 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:47.233927 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:47.234258 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:47.237288 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "bc35f601199341bfb65e63020f2bdd67"
format_stamp: "Formatted at 2026-08-07 22:49:47 on dist-test-slave-5k54"
server_key: "c5b6af6558d5f16786dab6ce88ba6248"
server_key_iv: "b071cbe091b6d8a0169b0d33b7257fb5"
server_key_version: "encryptionkey@0"
I20260807 22:49:47.238065 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "bc35f601199341bfb65e63020f2bdd67"
format_stamp: "Formatted at 2026-08-07 22:49:47 on dist-test-slave-5k54"
server_key: "c5b6af6558d5f16786dab6ce88ba6248"
server_key_iv: "b071cbe091b6d8a0169b0d33b7257fb5"
server_key_version: "encryptionkey@0"
I20260807 22:49:47.243238 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:49:47.246750 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:47.248106 10406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:47.248287 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "bc35f601199341bfb65e63020f2bdd67"
format_stamp: "Formatted at 2026-08-07 22:49:47 on dist-test-slave-5k54"
server_key: "c5b6af6558d5f16786dab6ce88ba6248"
server_key_iv: "b071cbe091b6d8a0169b0d33b7257fb5"
server_key_version: "encryptionkey@0"
I20260807 22:49:47.248494 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:47.260730 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:47.264642 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:47.264945 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:47.265885 10406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:47.267194 10406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:47.267302 10406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:47.267516 10406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:47.267596 10406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 22:49:47.302860 10406 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39423
I20260807 22:49:47.302907 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39423 every 8 connection(s)
I20260807 22:49:47.304270 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:49:47.309351 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10406
I20260807 22:49:47.309558 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:49:47.309998 9995 external_mini_cluster.cc:1428] Setting key ef9c854f72ffdb4dacf09ce4a2904862
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:47.324899 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.306886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:41253 (local address 127.9.194.254:44521)
0807 22:49:47.307062 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.307066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.308303 (+ 1237us) server_negotiation.cc:407] Connection header received
0807 22:49:47.310338 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.310343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.310441 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.310557 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.312361 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.313313 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.314936 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.315370 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.315535 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.318996 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.319001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.319005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.319036 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.321817 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.322532 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.322540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.322548 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.322673 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.323119 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.323127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.323130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.323394 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.323603 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.324291 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.324524 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:49:47.325977 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.307320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44521 (local address 127.9.194.195:41253)
0807 22:49:47.308118 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:47.308166 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:47.310019 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:47.310794 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:47.310802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:47.311528 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:47.312198 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:47.312215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.313486 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.313490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:47.314780 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:47.314797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.315635 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.315642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:47.315877 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.316542 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:47.316563 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:47.318775 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:47.321957 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:47.321966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:47.321983 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:47.322406 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:47.322807 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:47.322812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:47.322816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:47.323004 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:47.323573 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:47.323581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:47.323916 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 22:49:47.324194 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 22:49:47.324748 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":310,"threads_started":1}
I20260807 22:49:47.327548 10541 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44521
I20260807 22:49:47.327836 10541 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:47.328697 10541 heartbeater.cc:511] Master 127.9.194.254:44521 requested a full tablet report, sending...
I20260807 22:49:47.330139 10051 ts_manager.cc:194] Registered new tserver with Master: bc35f601199341bfb65e63020f2bdd67 (127.9.194.195:39423)
I20260807 22:49:47.330835 10256 heartbeater.cc:503] Master 127.9.194.254:44521 was elected leader, sending a full tablet report...
I20260807 22:49:47.331197 10051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:41253
I20260807 22:49:47.339520 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:47.389174 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.373106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.9.194.193:33847)
0807 22:49:47.373795 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.373806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.373968 (+ 162us) server_negotiation.cc:407] Connection header received
0807 22:49:47.374179 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.374186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.374449 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.374629 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.376258 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.377276 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.378557 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.379140 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.379285 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.383356 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.383367 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.383380 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.383415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.386291 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.386965 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.386972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.386975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.387103 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.387576 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.387580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.387582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.387930 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.388240 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.388549 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.388814 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":315,"threads_started":1}
W20260807 22:49:47.390537 10163 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60096
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:47.438987 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.425436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.9.194.193:33847)
0807 22:49:47.425671 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.425677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.425859 (+ 182us) server_negotiation.cc:407] Connection header received
0807 22:49:47.426008 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.426011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.426077 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.426166 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.427528 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.428296 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.429649 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.430121 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.430214 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.434161 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.434165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.434168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.434199 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.436768 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.437393 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.437397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.437399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.437517 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.437817 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.437820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.437822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.438047 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.438219 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.438572 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.438740 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:49:47.441002 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10118
I20260807 22:49:47.453361 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.442001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.9.194.193:33847)
0807 22:49:47.442154 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.442158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.442538 (+ 380us) server_negotiation.cc:407] Connection header received
0807 22:49:47.442650 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.442653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.442706 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.442782 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.444023 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.444683 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.445905 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.446324 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.446413 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.448562 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.448566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.448569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.448597 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.450971 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.451626 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.451629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.451631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.451779 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.452112 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.452116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.452119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.452369 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.452555 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.452833 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.452992 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:49:47.454116 10251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:47.542052 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10118
I20260807 22:49:47.561092 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10262
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:49:47.576511 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.562021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.9.194.194:38245)
0807 22:49:47.562511 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.562519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.562543 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:49:47.562602 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.562608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.562815 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.562938 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.564151 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.565136 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.566294 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.566813 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.566906 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.570876 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.570886 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.570900 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.570931 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.573709 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.574311 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.574317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.574320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.574435 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.574804 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.574808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.574830 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.575184 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.575510 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.575852 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.576096 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":249,"threads_started":1}
I20260807 22:49:47.577755 10395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:47.662839 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10262
I20260807 22:49:47.680996 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10406
Aug 07 22:49:47 dist-test-slave-5k54 krb5kdc[10003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:49:47.697504 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:47.682133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.9.194.195:39423)
0807 22:49:47.682836 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:47.682843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:47.682869 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:49:47.682929 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:47.682935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:47.683149 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:47.683284 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:47.684590 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.685643 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.686852 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:47.687396 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:47.687682 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:47.691940 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:47.691950 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:47.691977 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:47.692010 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:47.694839 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.695628 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.695638 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.695641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.695784 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:47.696107 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:47.696110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:47.696112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:47.696441 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:47.696673 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:47.696908 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:49:47.697113 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":385,"threads_started":1}
I20260807 22:49:47.698827 10536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:47.781718 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10406
I20260807 22:49:47.800390 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10016
I20260807 22:49:47.801607 10085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:47.904310 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10016
2026-08-07T22:49:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4131 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:48 dist-test-slave-5k54 krb5kdc[10562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:48 dist-test-slave-5k54 krb5kdc[10562](info): set up 2 sockets
Aug 07 22:49:48 dist-test-slave-5k54 krb5kdc[10562](info): commencing operation
krb5kdc: starting...
W20260807 22:49:49.045884 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:49:49 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:49:49.237186 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44671
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:49.556581 10576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:49.556968 10576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:49.557147 10576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:49.564380 10576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:49.564535 10576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:49.564651 10576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:49.564760 10576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:49.564888 10576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:49.574736 10576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:44671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:49.576149 10576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:49.577586 10576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:49.589687 10581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:49.589716 10584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:49.589751 10582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:49.590413 10576 server_base.cc:1034] running on GCE node
I20260807 22:49:49.591205 10576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:49.592677 10576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:49.593899 10576 hybrid_clock.cc:648] HybridClock initialized: now 1786142989593844 us; error 32 us; skew 500 ppm
I20260807 22:49:49.594255 10576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:49 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:49.603046 10576 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:49:49.604720 10576 webserver.cc:460] Webserver started at http://127.9.194.254:37415/ using document root <none> and password file <none>
I20260807 22:49:49.605482 10576 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:49.605631 10576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:49.605957 10576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:49.609501 10576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "4e1dbbdda2b842c0ad00655f9b16ed52"
format_stamp: "Formatted at 2026-08-07 22:49:49 on dist-test-slave-5k54"
server_key: "6eeadb9867dc602a7eb4b3349a1f98a4"
server_key_iv: "7506eac660380bd6fcb9fa49867e7ebd"
server_key_version: "encryptionkey@0"
I20260807 22:49:49.610409 10576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "4e1dbbdda2b842c0ad00655f9b16ed52"
format_stamp: "Formatted at 2026-08-07 22:49:49 on dist-test-slave-5k54"
server_key: "6eeadb9867dc602a7eb4b3349a1f98a4"
server_key_iv: "7506eac660380bd6fcb9fa49867e7ebd"
server_key_version: "encryptionkey@0"
I20260807 22:49:49.615617 10576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:49:49.619051 10592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:49.620432 10576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:49.620637 10576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "4e1dbbdda2b842c0ad00655f9b16ed52"
format_stamp: "Formatted at 2026-08-07 22:49:49 on dist-test-slave-5k54"
server_key: "6eeadb9867dc602a7eb4b3349a1f98a4"
server_key_iv: "7506eac660380bd6fcb9fa49867e7ebd"
server_key_version: "encryptionkey@0"
I20260807 22:49:49.620834 10576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:49.640920 10576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:49.645212 10576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:49.645510 10576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:49.665470 10576 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:44671
I20260807 22:49:49.665493 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:44671 every 8 connection(s)
I20260807 22:49:49.666986 10576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:49.670814 10647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:49.675258 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10576
I20260807 22:49:49.675535 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:49:49.676388 9995 external_mini_cluster.cc:1428] Setting key 44c0f1b24df64a00549e991eb035b28e
I20260807 22:49:49.682677 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52: Bootstrap starting.
I20260807 22:49:49.686347 10647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:49.687582 10647 log.cc:824] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:49.690981 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52: No bootstrap required, opened a new log
Aug 07 22:49:49 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:49.701521 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:49.678430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52864 (local address 127.9.194.254:44671)
0807 22:49:49.679250 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:49.679261 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:49.679305 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:49:49.681238 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:49.681262 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:49.681630 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:49.681950 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:49.683741 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:49.685079 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:49.686809 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:49.687573 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:49.687731 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:49.692695 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:49.692710 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:49.692732 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:49.692780 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:49.696958 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:49.697632 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:49.697645 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:49.697654 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:49.697855 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:49.698204 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:49.698209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:49.698213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:49.698935 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:49.699202 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:49.699671 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 22:49:49.700161 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":373,"threads_started":1}
I20260807 22:49:49.701972 10647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } }
I20260807 22:49:49.702348 10647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:49.702471 10647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1dbbdda2b842c0ad00655f9b16ed52, State: Initialized, Role: FOLLOWER
I20260807 22:49:49.703115 10647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } }
I20260807 22:49:49.703368 10647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:49.703562 10647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:49.703763 10647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:49.706259 10647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } }
I20260807 22:49:49.706745 10647 leader_election.cc:302] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e1dbbdda2b842c0ad00655f9b16ed52; no voters:
I20260807 22:49:49.707393 10647 leader_election.cc:288] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:49.707538 10652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:49.708508 10652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [term 1 LEADER]: Becoming Leader. State: Replica: 4e1dbbdda2b842c0ad00655f9b16ed52, State: Running, Role: LEADER
I20260807 22:49:49.709009 10652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } }
I20260807 22:49:49.709738 10647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:49.712127 10653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } } }
I20260807 22:49:49.712757 10653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:49.712715 10654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1dbbdda2b842c0ad00655f9b16ed52. Latest consensus state: current_term: 1 leader_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1dbbdda2b842c0ad00655f9b16ed52" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44671 } } }
I20260807 22:49:49.713172 10654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:49.714987 10661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:49.719625 10661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:49.728180 10661 catalog_manager.cc:1347] Generated new cluster ID: e6096de6e6e64d6b90b06124df83d1b8
I20260807 22:49:49.728310 10661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:49.751121 10661 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:49.753140 10661 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:49.763554 10661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e1dbbdda2b842c0ad00655f9b16ed52: Generated new TSK 0
I20260807 22:49:49.764591 10661 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:49:49.819885 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:50.147943 10674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:50.148218 10674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:50.148316 10674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:50.155843 10674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:50.155977 10674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:50.156095 10674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:49:50.165904 10674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:50.167214 10674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:50.168571 10674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:50.181488 10679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:50.181656 10680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:50.181798 10682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:50.182418 10674 server_base.cc:1034] running on GCE node
I20260807 22:49:50.182996 10674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:50.183990 10674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:50.185220 10674 hybrid_clock.cc:648] HybridClock initialized: now 1786142990185188 us; error 59 us; skew 500 ppm
I20260807 22:49:50.185551 10674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:50 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:50.188876 10674 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:49:50.190368 10674 webserver.cc:460] Webserver started at http://127.9.194.193:41005/ using document root <none> and password file <none>
I20260807 22:49:50.191063 10674 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:50.191190 10674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:50.191593 10674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:50.194428 10674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "cb067775bca1ec6782c7aace94f51acb"
server_key_iv: "e2efeb459ac74dc91411dd106334e72f"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.195222 10674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "cb067775bca1ec6782c7aace94f51acb"
server_key_iv: "e2efeb459ac74dc91411dd106334e72f"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.200592 10674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:49:50.204085 10690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.205408 10674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:49:50.205539 10674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "cb067775bca1ec6782c7aace94f51acb"
server_key_iv: "e2efeb459ac74dc91411dd106334e72f"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.205695 10674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:50.223896 10674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:50.228406 10674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:50.228690 10674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:50.229655 10674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:50.230979 10674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:50.231068 10674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.231232 10674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:50.231295 10674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.264087 10674 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:42805
I20260807 22:49:50.264164 10812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:42805 every 8 connection(s)
I20260807 22:49:50.265441 10674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:49:50.269243 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10674
I20260807 22:49:50.269501 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:49:50.270337 9995 external_mini_cluster.cc:1428] Setting key e12c5d5f968bc64da8ed80e4bedf30e1
Aug 07 22:49:50 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:50.286700 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:50.268158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:45801 (local address 127.9.194.254:44671)
0807 22:49:50.268589 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:50.268594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:50.269608 (+ 1014us) server_negotiation.cc:407] Connection header received
0807 22:49:50.272082 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:50.272088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:50.272176 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:50.272290 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:50.273736 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.274473 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.276542 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.276947 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.277079 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:50.280852 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:50.280857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:50.280860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:50.280891 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:50.283597 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:50.284412 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:50.284419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:50.284422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:50.284550 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:50.284999 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:50.285004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:50.285007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:50.285268 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:50.285422 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:50.286106 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 22:49:50.286335 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
I20260807 22:49:50.287681 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:50.268583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44671 (local address 127.9.194.193:45801)
0807 22:49:50.269427 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:50.269491 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:50.271817 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:50.272523 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:50.272531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:50.273034 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:50.273574 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:50.273590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.274640 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.274644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:50.276383 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:50.276396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.277181 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.277190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:50.277423 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:50.278099 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:50.278124 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:50.280627 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:50.283753 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:50.283761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:50.283778 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:50.284278 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:50.284663 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:50.284668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:50.284672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:50.284875 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:50.285397 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:50.285412 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:50.285737 (+ 325us) client_negotiation.cc:769] Sending connection context
0807 22:49:50.285999 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 22:49:50.286534 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":308,"threads_started":1}
I20260807 22:49:50.289141 10813 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44671
I20260807 22:49:50.289461 10813 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:50.290226 10813 heartbeater.cc:511] Master 127.9.194.254:44671 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
I20260807 22:49:50.292687 10610 ts_manager.cc:194] Registered new tserver with Master: c9a6bf3eaaa04f7dbd1a685afc37ad0f (127.9.194.193:42805)
I20260807 22:49:50.294857 10610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:45801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:49:50.314184 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:50.636302 10820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:50.636623 10820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:50.636807 10820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:50.643987 10820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:50.644162 10820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:50.644361 10820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:49:50.654130 10820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:50.655393 10820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:50.656770 10820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:50.669912 10828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:50.669863 10825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:50.669833 10826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:50.672978 10820 server_base.cc:1034] running on GCE node
I20260807 22:49:50.673619 10820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:50.674584 10820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:50.675809 10820 hybrid_clock.cc:648] HybridClock initialized: now 1786142990675758 us; error 44 us; skew 500 ppm
I20260807 22:49:50.676121 10820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:50 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:50.679293 10820 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:49:50.680908 10820 webserver.cc:460] Webserver started at http://127.9.194.194:41725/ using document root <none> and password file <none>
I20260807 22:49:50.681603 10820 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:50.681726 10820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:50.682201 10820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:50.685389 10820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "53d6e809ad7d611f0bc4b8b06f7ad160"
server_key_iv: "df4a7bbc4415cf79c95f5700df3e86aa"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.686203 10820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "53d6e809ad7d611f0bc4b8b06f7ad160"
server_key_iv: "df4a7bbc4415cf79c95f5700df3e86aa"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.691319 10820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 22:49:50.694756 10836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.696192 10820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:50.696360 10820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41"
format_stamp: "Formatted at 2026-08-07 22:49:50 on dist-test-slave-5k54"
server_key: "53d6e809ad7d611f0bc4b8b06f7ad160"
server_key_iv: "df4a7bbc4415cf79c95f5700df3e86aa"
server_key_version: "encryptionkey@0"
I20260807 22:49:50.696534 10820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:50.716368 10820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:50.720729 10820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:50.721035 10820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:50.722025 10820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:50.723349 10820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:50.723464 10820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.723623 10820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:50.723675 10820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:50.757071 10820 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:41277
I20260807 22:49:50.757112 10957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:41277 every 8 connection(s)
I20260807 22:49:50.758400 10820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:49:50.761895 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10820
I20260807 22:49:50.762107 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:49:50.762636 9995 external_mini_cluster.cc:1428] Setting key 79fcc22387574b3521ee929a4550fb4a
Aug 07 22:49:50 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:50.781355 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:50.760981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59921 (local address 127.9.194.254:44671)
0807 22:49:50.761211 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:50.761218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:50.762452 (+ 1234us) server_negotiation.cc:407] Connection header received
0807 22:49:50.764637 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:50.764643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:50.764734 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:50.764866 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:50.766337 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.767080 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.769739 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.770194 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.770337 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:50.773996 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:50.774005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:50.774013 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:50.774061 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:50.777774 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:50.778684 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:50.778692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:50.778699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:50.778868 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:50.779315 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:50.779322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:50.779327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:50.779778 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:50.779969 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:50.780654 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 22:49:50.780941 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:49:50.782155 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:50.761460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44671 (local address 127.9.194.194:59921)
0807 22:49:50.762273 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:50.762325 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:50.764360 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:50.765101 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:50.765109 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:50.765620 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:50.766169 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:50.766185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.767261 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.767269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:50.769593 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:50.769611 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:50.770420 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:50.770425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:50.770620 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:50.771320 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:50.771346 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:50.773744 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:50.777969 (+ 4225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:50.777976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:50.777990 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:50.778538 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:50.778993 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:50.778997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:50.778999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:50.779180 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:50.779913 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:50.779920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:50.780239 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 22:49:50.780482 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 22:49:50.780916 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:49:50.783766 10958 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44671
I20260807 22:49:50.784096 10958 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:50.784829 10958 heartbeater.cc:511] Master 127.9.194.254:44671 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
I20260807 22:49:50.786429 10610 ts_manager.cc:194] Registered new tserver with Master: 5eb0ed67c6ba4c19a5c9bfa986437e41 (127.9.194.194:41277)
I20260807 22:49:50.787271 10610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:59921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:49:50.810425 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:51.136291 10963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:51.136569 10963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:51.136667 10963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:51.143688 10963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:51.143795 10963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:51.143915 10963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:49:51.153473 10963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:51.154706 10963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:51.155997 10963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:51.169112 10968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:51.169155 10969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:51.169185 10971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:51.170256 10963 server_base.cc:1034] running on GCE node
I20260807 22:49:51.170768 10963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:51.171633 10963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:51.172856 10963 hybrid_clock.cc:648] HybridClock initialized: now 1786142991172823 us; error 52 us; skew 500 ppm
I20260807 22:49:51.173169 10963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:51.176278 10963 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:49:51.177855 10963 webserver.cc:460] Webserver started at http://127.9.194.195:34969/ using document root <none> and password file <none>
I20260807 22:49:51.178572 10963 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:51.178720 10963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:51.179057 10963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:51.181949 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "b58b5bc6600048f9bfeff6db29a0fb23"
format_stamp: "Formatted at 2026-08-07 22:49:51 on dist-test-slave-5k54"
server_key: "c6f5f7a8081df286d5786998a1929524"
server_key_iv: "a3a15ed1f90ea6e6d21ea000ec932b17"
server_key_version: "encryptionkey@0"
I20260807 22:49:51.182732 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "b58b5bc6600048f9bfeff6db29a0fb23"
format_stamp: "Formatted at 2026-08-07 22:49:51 on dist-test-slave-5k54"
server_key: "c6f5f7a8081df286d5786998a1929524"
server_key_iv: "a3a15ed1f90ea6e6d21ea000ec932b17"
server_key_version: "encryptionkey@0"
I20260807 22:49:51.188150 10963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 22:49:51.191578 10978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:51.192859 10963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:49:51.193045 10963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "b58b5bc6600048f9bfeff6db29a0fb23"
format_stamp: "Formatted at 2026-08-07 22:49:51 on dist-test-slave-5k54"
server_key: "c6f5f7a8081df286d5786998a1929524"
server_key_iv: "a3a15ed1f90ea6e6d21ea000ec932b17"
server_key_version: "encryptionkey@0"
I20260807 22:49:51.193240 10963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:51.211385 10963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:51.215337 10963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:51.215689 10963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:51.216652 10963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:49:51.218091 10963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:51.218194 10963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:51.218343 10963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:51.218417 10963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:51.252432 10963 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:32783
I20260807 22:49:51.252489 11094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:32783 every 8 connection(s)
I20260807 22:49:51.253827 10963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:49:51.258744 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 10963
I20260807 22:49:51.258930 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:49:51.259368 9995 external_mini_cluster.cc:1428] Setting key ecdfdd822237d8acff5243b28bb8bf0e
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:51.275024 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.256374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:45621 (local address 127.9.194.254:44671)
0807 22:49:51.256605 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.256612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.257741 (+ 1129us) server_negotiation.cc:407] Connection header received
0807 22:49:51.259704 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.259710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.259802 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.259926 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.261670 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.262712 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.264305 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.264991 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.265110 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.268964 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.268969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.268972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.269009 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.271687 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.272400 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.272405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.272407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.272545 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.273019 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.273023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.273024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.273418 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.273576 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.274455 (+ 879us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.274673 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:49:51.275983 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.256886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44671 (local address 127.9.194.195:45621)
0807 22:49:51.257580 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.257619 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.259373 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.260226 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.260244 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.260853 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:51.261521 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.261539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.262847 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.262851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.264158 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.264170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.265195 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.265200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.265399 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.266077 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:51.266103 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:51.268740 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:51.271815 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:51.271821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:51.271832 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:51.272250 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:51.272643 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:51.272646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:51.272648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:51.272910 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:51.273550 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:51.273558 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:51.273984 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.274315 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.274832 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":269,"threads_started":1}
I20260807 22:49:51.277220 11095 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44671
I20260807 22:49:51.277532 11095 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:51.278235 11095 heartbeater.cc:511] Master 127.9.194.254:44671 requested a full tablet report, sending...
I20260807 22:49:51.279626 10610 ts_manager.cc:194] Registered new tserver with Master: b58b5bc6600048f9bfeff6db29a0fb23 (127.9.194.195:32783)
I20260807 22:49:51.280411 10610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:45621
I20260807 22:49:51.287001 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:49:51.298462 10813 heartbeater.cc:503] Master 127.9.194.254:44671 was elected leader, sending a full tablet report...
I20260807 22:49:51.306526 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.293707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.9.194.254:44671)
0807 22:49:51.293897 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.293905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.294429 (+ 524us) server_negotiation.cc:407] Connection header received
0807 22:49:51.294569 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.294572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.294632 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.294709 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.296244 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.297022 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.298330 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.298925 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.299044 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.301492 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.301496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.301499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.301528 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.304301 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.304890 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.304894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.304896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.305016 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.305320 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.305323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.305325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.305610 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.305794 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.306045 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.306193 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:49:51.314662 10610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:49:51.322034 10610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:49:51.359411 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.337412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.9.194.193:42805)
0807 22:49:51.349688 (+ 12276us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.349701 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.349743 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:49:51.349880 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.349890 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.350238 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.352298 (+ 2060us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:51.352598 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.354448 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.356809 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.358242 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.358397 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.358645 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.358901 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.359064 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12036,"thread_start_us":13744,"threads_started":1}
I20260807 22:49:51.359579 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.337101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42805 (local address 127.0.0.1:37492)
0807 22:49:51.338023 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.338073 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.338300 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.351550 (+ 13250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.351562 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.351612 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:51.352051 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.352069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.354633 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.354638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.356657 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.356669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.358448 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.358455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.358644 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.358702 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.358850 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.359042 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":63,"thread_start_us":14977,"threads_started":1}
I20260807 22:49:51.360316 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.337273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:32783 (local address 127.0.0.1:49058)
0807 22:49:51.350989 (+ 13716us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.351015 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.351542 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.352385 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.352389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.352416 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:51.352853 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.352866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.354788 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.354794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.357228 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.357243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.359379 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.359388 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.359618 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.359648 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.359770 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.359908 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13543,"mutex_wait_us":97,"thread_start_us":920,"threads_started":1}
I20260807 22:49:51.360328 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.337875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49058 (local address 127.9.194.195:32783)
0807 22:49:51.338962 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.338970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.351135 (+ 12165us) server_negotiation.cc:407] Connection header received
0807 22:49:51.351773 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.351782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.352040 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.352220 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:51.353063 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.354662 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.357423 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.358865 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.359004 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.359220 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.359829 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.360003 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":1364,"threads_started":1}
I20260807 22:49:51.362257 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.339158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41277 (local address 127.0.0.1:42872)
0807 22:49:51.352364 (+ 13206us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.352384 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.352523 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.353122 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.353126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.353151 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:51.353545 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.353557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.355532 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.355540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.357782 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.357797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.361417 (+ 3620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.361427 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.361610 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.361640 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.361740 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.361864 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13078,"mutex_wait_us":45,"thread_start_us":11505,"threads_started":1}
I20260807 22:49:51.362854 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.339711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.9.194.194:41277)
0807 22:49:51.351985 (+ 12274us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.351994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.352525 (+ 531us) server_negotiation.cc:407] Connection header received
0807 22:49:51.352711 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.352720 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.352933 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.353142 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:51.353741 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.355362 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.357971 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.359471 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.360070 (+ 599us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.360264 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.362378 (+ 2114us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.362539 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12108,"thread_start_us":11988,"threads_started":1}
I20260807 22:49:51.364228 10739 tablet_service.cc:1467] Processing CreateTablet for tablet 2547e55845dc49e49635390daf603a5c (DEFAULT_TABLE table=test-table [id=043f5fa102fb4e35b6dab43a5e204830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:51.364305 11028 tablet_service.cc:1467] Processing CreateTablet for tablet 2547e55845dc49e49635390daf603a5c (DEFAULT_TABLE table=test-table [id=043f5fa102fb4e35b6dab43a5e204830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:51.365690 10739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2547e55845dc49e49635390daf603a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:51.365767 11028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2547e55845dc49e49635390daf603a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:51.366367 10887 tablet_service.cc:1467] Processing CreateTablet for tablet 2547e55845dc49e49635390daf603a5c (DEFAULT_TABLE table=test-table [id=043f5fa102fb4e35b6dab43a5e204830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:51.367722 10887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2547e55845dc49e49635390daf603a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:51.377460 11116 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Bootstrap starting.
I20260807 22:49:51.380100 11117 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Bootstrap starting.
I20260807 22:49:51.381908 11116 tablet_bootstrap.cc:654] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:51.383234 11116 log.cc:824] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:51.383579 11117 tablet_bootstrap.cc:654] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:51.384941 11117 log.cc:824] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:51.386482 11116 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: No bootstrap required, opened a new log
I20260807 22:49:51.386754 11116 ts_tablet_manager.cc:1397] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 22:49:51.387593 11118 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Bootstrap starting.
I20260807 22:49:51.388118 11117 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: No bootstrap required, opened a new log
I20260807 22:49:51.388392 11117 ts_tablet_manager.cc:1397] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 22:49:51.391003 11118 tablet_bootstrap.cc:654] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:51.392086 11118 log.cc:824] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:51.394516 11118 tablet_bootstrap.cc:492] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: No bootstrap required, opened a new log
I20260807 22:49:51.394774 11118 ts_tablet_manager.cc:1397] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 22:49:51.397924 11116 raft_consensus.cc:348] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.398355 11116 raft_consensus.cc:374] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:51.398464 11116 raft_consensus.cc:729] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9a6bf3eaaa04f7dbd1a685afc37ad0f, State: Initialized, Role: FOLLOWER
I20260807 22:49:51.399075 11116 consensus_queue.cc:260] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.398976 11117 raft_consensus.cc:348] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.399385 11117 raft_consensus.cc:374] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:51.399562 11117 raft_consensus.cc:729] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58b5bc6600048f9bfeff6db29a0fb23, State: Initialized, Role: FOLLOWER
I20260807 22:49:51.400343 11117 consensus_queue.cc:260] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.403301 11118 raft_consensus.cc:348] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.403726 11118 raft_consensus.cc:374] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:51.403851 11118 raft_consensus.cc:729] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb0ed67c6ba4c19a5c9bfa986437e41, State: Initialized, Role: FOLLOWER
I20260807 22:49:51.404399 11118 consensus_queue.cc:260] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.405900 10958 heartbeater.cc:503] Master 127.9.194.254:44671 was elected leader, sending a full tablet report...
I20260807 22:49:51.406385 11118 ts_tablet_manager.cc:1428] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260807 22:49:51.409092 11116 ts_tablet_manager.cc:1428] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Time spent starting tablet: real 0.022s user 0.012s sys 0.008s
I20260807 22:49:51.410061 11095 heartbeater.cc:503] Master 127.9.194.254:44671 was elected leader, sending a full tablet report...
I20260807 22:49:51.410930 11117 ts_tablet_manager.cc:1428] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Time spent starting tablet: real 0.022s user 0.010s sys 0.012s
W20260807 22:49:51.505496 11096 tablet.cc:2369] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:49:51.510505 10959 tablet.cc:2369] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:49:51.517858 10815 tablet.cc:2369] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:51.609192 11125 raft_consensus.cc:482] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:49:51.609534 11125 raft_consensus.cc:504] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.611891 11125 leader_election.cc:288] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9a6bf3eaaa04f7dbd1a685afc37ad0f (127.9.194.193:42805), b58b5bc6600048f9bfeff6db29a0fb23 (127.9.194.195:32783)
I20260807 22:49:51.618798 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.612474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:35453 (local address 127.9.194.195:32783)
0807 22:49:51.612699 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.612705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.612976 (+ 271us) server_negotiation.cc:407] Connection header received
0807 22:49:51.613212 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.613216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.613280 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.613379 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:51.613987 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.614791 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.616994 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.617788 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.617981 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.618059 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.618407 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.618532 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 22:49:51.618925 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.612310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59351 (local address 127.9.194.193:42805)
0807 22:49:51.612545 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.612554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.613535 (+ 981us) server_negotiation.cc:407] Connection header received
0807 22:49:51.613755 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.613762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.613830 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.613927 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:51.614481 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.615235 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.617130 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.617931 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.618122 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.618206 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.618554 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.618677 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:49:51.619058 11127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.612146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42805 (local address 127.9.194.194:59351)
0807 22:49:51.613416 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.613431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.613617 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.613939 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.613944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.613966 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:51.614294 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.614302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.615401 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.615405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.616970 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.616981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.618154 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.618160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.618389 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.618411 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.618519 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.618643 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"mutex_wait_us":679,"thread_start_us":334,"threads_started":1}
I20260807 22:49:51.619338 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.612146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:32783 (local address 127.9.194.194:35453)
0807 22:49:51.612858 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:51.612873 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:51.613055 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:51.613409 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:51.613414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:51.613443 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:51.613837 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.613846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.614923 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.614931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.616838 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:51.616850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.618006 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.618013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:51.618203 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.618268 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 22:49:51.618389 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:49:51.618516 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":109,"spinlock_wait_cycles":17664,"thread_start_us":399,"threads_started":1}
I20260807 22:49:51.620397 11049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2547e55845dc49e49635390daf603a5c" candidate_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" is_pre_election: true
I20260807 22:49:51.620400 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2547e55845dc49e49635390daf603a5c" candidate_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" is_pre_election: true
I20260807 22:49:51.621052 10759 raft_consensus.cc:2454] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb0ed67c6ba4c19a5c9bfa986437e41 in term 0.
I20260807 22:49:51.621760 10837 leader_election.cc:302] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb0ed67c6ba4c19a5c9bfa986437e41, c9a6bf3eaaa04f7dbd1a685afc37ad0f; no voters:
I20260807 22:49:51.622105 11125 raft_consensus.cc:2781] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:49:51.622284 11125 raft_consensus.cc:482] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:49:51.622404 11125 raft_consensus.cc:3037] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:51.622750 11049 raft_consensus.cc:2454] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb0ed67c6ba4c19a5c9bfa986437e41 in term 0.
I20260807 22:49:51.625382 11125 raft_consensus.cc:504] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.626016 11125 leader_election.cc:288] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [CANDIDATE]: Term 1 election: Requested vote from peers c9a6bf3eaaa04f7dbd1a685afc37ad0f (127.9.194.193:42805), b58b5bc6600048f9bfeff6db29a0fb23 (127.9.194.195:32783)
I20260807 22:49:51.626569 11049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2547e55845dc49e49635390daf603a5c" candidate_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58b5bc6600048f9bfeff6db29a0fb23"
I20260807 22:49:51.626562 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2547e55845dc49e49635390daf603a5c" candidate_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f"
I20260807 22:49:51.626801 11049 raft_consensus.cc:3037] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:51.626859 10759 raft_consensus.cc:3037] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:51.630440 11049 raft_consensus.cc:2454] T 2547e55845dc49e49635390daf603a5c P b58b5bc6600048f9bfeff6db29a0fb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb0ed67c6ba4c19a5c9bfa986437e41 in term 1.
I20260807 22:49:51.630440 10759 raft_consensus.cc:2454] T 2547e55845dc49e49635390daf603a5c P c9a6bf3eaaa04f7dbd1a685afc37ad0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb0ed67c6ba4c19a5c9bfa986437e41 in term 1.
I20260807 22:49:51.630883 10837 leader_election.cc:302] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb0ed67c6ba4c19a5c9bfa986437e41, c9a6bf3eaaa04f7dbd1a685afc37ad0f; no voters:
I20260807 22:49:51.631198 11125 raft_consensus.cc:2781] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:51.631713 11125 raft_consensus.cc:686] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [term 1 LEADER]: Becoming Leader. State: Replica: 5eb0ed67c6ba4c19a5c9bfa986437e41, State: Running, Role: LEADER
I20260807 22:49:51.632222 11125 consensus_queue.cc:237] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } }
I20260807 22:49:51.637843 10606 catalog_manager.cc:5495] T 2547e55845dc49e49635390daf603a5c P 5eb0ed67c6ba4c19a5c9bfa986437e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5eb0ed67c6ba4c19a5c9bfa986437e41 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9a6bf3eaaa04f7dbd1a685afc37ad0f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b58b5bc6600048f9bfeff6db29a0fb23" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eb0ed67c6ba4c19a5c9bfa986437e41" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41277 } health_report { overall_health: HEALTHY } } }
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:51.661723 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.646758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.9.194.193:42805)
0807 22:49:51.646915 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.646919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.647456 (+ 537us) server_negotiation.cc:407] Connection header received
0807 22:49:51.647616 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.647620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.647696 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.647777 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.649282 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.650056 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.651449 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.651890 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.652022 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.656058 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.656072 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.656088 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.656120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.658915 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.659618 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.659632 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.659634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.659773 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.660163 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.660167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.660169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.660522 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.660816 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.661149 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.661380 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:51.715214 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.700323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.9.194.193:42805)
0807 22:49:51.700616 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.700626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.700974 (+ 348us) server_negotiation.cc:407] Connection header received
0807 22:49:51.701119 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.701123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.701190 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.701366 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.702629 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.703349 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.705057 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.705472 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.705567 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.709953 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.709959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.709962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.710019 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.712753 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.713390 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.713394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.713396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.713517 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.713962 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.713966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.713968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.714260 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.714503 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.714716 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.714893 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:51.763533 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.748962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.9.194.193:42805)
0807 22:49:51.749238 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.749245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.749538 (+ 293us) server_negotiation.cc:407] Connection header received
0807 22:49:51.749757 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.749761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.749831 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.749951 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.751368 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.752123 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.753344 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.753795 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.753958 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.758304 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.758309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.758312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.758345 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.760952 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.761728 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.761732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.761734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.761861 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.762222 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.762225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.762227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.762514 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.762741 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.763044 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.763224 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260807 22:49:51.765637 10718 tablet_service.cc:2908] Rejecting scan request for tablet 2547e55845dc49e49635390daf603a5c: Uninitialized: clean time has not yet been initialized
I20260807 22:49:51.769305 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10674
I20260807 22:49:51.782698 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.770738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.9.194.193:42805)
0807 22:49:51.770949 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.770969 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.771300 (+ 331us) server_negotiation.cc:407] Connection header received
0807 22:49:51.771514 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.771518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.771588 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.771666 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.772927 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.773600 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.774740 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.775133 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.775235 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.777742 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.777747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.777750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.777778 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.780428 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.781064 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.781068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.781070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.781176 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.781518 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.781521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.781523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.781784 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.781993 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.782250 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.782395 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:49:51.783427 10808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:51.877295 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10674
I20260807 22:49:51.897135 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10820
Aug 07 22:49:51 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:49:51.912505 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:51.898039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.9.194.194:41277)
0807 22:49:51.898460 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:51.898466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:51.898489 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:49:51.898543 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:51.898547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:51.898610 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:51.898705 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:51.900034 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.900777 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.902080 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:51.902470 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:51.902563 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:51.906789 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:51.906802 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:51.906820 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:51.906864 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:51.909736 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.910414 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.910422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.910424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.910565 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:51.910944 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:51.910947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:51.910949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:51.911333 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:51.911603 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:51.911934 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 22:49:51.912115 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260807 22:49:51.913115 10953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:52.002290 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10820
I20260807 22:49:52.021624 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10963
Aug 07 22:49:52 dist-test-slave-5k54 krb5kdc[10562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:49:52.037178 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:52.022416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.9.194.195:32783)
0807 22:49:52.022592 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:52.022597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:52.022616 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:49:52.022793 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:52.022797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:52.022872 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:52.022957 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:52.024335 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:52.025128 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:52.026576 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:52.027151 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:52.027295 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:52.032048 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:52.032062 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:52.032079 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:52.032112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:52.034874 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:52.035517 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:52.035525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:52.035527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:52.035671 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:52.036014 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:52.036017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:52.036019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:52.036334 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:52.036559 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:52.036745 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 22:49:52.036906 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:49:52.037842 11090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:52.125226 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10963
I20260807 22:49:52.143522 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 10576
I20260807 22:49:52.144755 10642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:52.254763 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 10576
2026-08-07T22:49:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4351 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:52 dist-test-slave-5k54 krb5kdc[11152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:52 dist-test-slave-5k54 krb5kdc[11152](info): set up 2 sockets
Aug 07 22:49:52 dist-test-slave-5k54 krb5kdc[11152](info): commencing operation
krb5kdc: starting...
W20260807 22:49:53.384397 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:49:53 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:49:53.580940 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39961
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:39961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:53.900652 11165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:53.900967 11165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:53.901134 11165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:53.908283 11165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:53.908390 11165 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:49:53.908494 11165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:53.908608 11165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:53.908685 11165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:53.908775 11165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:53.918654 11165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:39961
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:53.920046 11165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:53.921424 11165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:53.933332 11170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:53.933439 11171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:53.933518 11173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:53.934566 11165 server_base.cc:1034] running on GCE node
I20260807 22:49:53.935336 11165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:53.936760 11165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:53.938038 11165 hybrid_clock.cc:648] HybridClock initialized: now 1786142993937956 us; error 41 us; skew 500 ppm
I20260807 22:49:53.938407 11165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:53 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142993, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:53.941642 11165 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:49:53.943158 11165 webserver.cc:460] Webserver started at http://127.9.194.254:39437/ using document root <none> and password file <none>
I20260807 22:49:53.943910 11165 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:53.944001 11165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:53.944267 11165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:53.947196 11165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "a499f5b7a64f478992aa6eaf64fc4cc8"
format_stamp: "Formatted at 2026-08-07 22:49:53 on dist-test-slave-5k54"
server_key: "939c8340f8d8d49832113328621ac81e"
server_key_iv: "7ca1ddf4d2350d88d7f6c2378cd55286"
server_key_version: "encryptionkey@0"
I20260807 22:49:53.948005 11165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "a499f5b7a64f478992aa6eaf64fc4cc8"
format_stamp: "Formatted at 2026-08-07 22:49:53 on dist-test-slave-5k54"
server_key: "939c8340f8d8d49832113328621ac81e"
server_key_iv: "7ca1ddf4d2350d88d7f6c2378cd55286"
server_key_version: "encryptionkey@0"
I20260807 22:49:53.953258 11165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:49:53.956617 11180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:53.957970 11165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:49:53.958163 11165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "a499f5b7a64f478992aa6eaf64fc4cc8"
format_stamp: "Formatted at 2026-08-07 22:49:53 on dist-test-slave-5k54"
server_key: "939c8340f8d8d49832113328621ac81e"
server_key_iv: "7ca1ddf4d2350d88d7f6c2378cd55286"
server_key_version: "encryptionkey@0"
I20260807 22:49:53.958405 11165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:53.985190 11165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:53.989769 11165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:53.989992 11165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:54.012171 11165 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:39961
I20260807 22:49:54.012204 11245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:39961 every 8 connection(s)
I20260807 22:49:54.013574 11165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:54.017279 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:54.019950 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11165
I20260807 22:49:54.020179 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:49:54.021008 9995 external_mini_cluster.cc:1428] Setting key b9b6a96ad2f2feb2183b19024830e234
I20260807 22:49:54.029963 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Bootstrap starting.
I20260807 22:49:54.033772 11246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:54.035157 11246 log.cc:824] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:49:54 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:54.038766 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: No bootstrap required, opened a new log
I20260807 22:49:54.045995 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:54.023147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.9.194.254:39961)
0807 22:49:54.024029 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:54.024043 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:54.024086 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:49:54.025749 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:54.025769 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:54.026159 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:54.026483 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:54.028242 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.029468 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.031140 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.031892 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.032034 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:54.037340 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:54.037354 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:54.037373 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:54.037422 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:54.041490 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.042157 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.042165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.042173 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.042388 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.042731 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.042736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.042739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.043479 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:54.043773 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:54.044139 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 22:49:54.044607 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":464,"threads_started":1}
I20260807 22:49:54.049392 11246 raft_consensus.cc:348] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } }
I20260807 22:49:54.049710 11246 raft_consensus.cc:374] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:54.049780 11246 raft_consensus.cc:729] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a499f5b7a64f478992aa6eaf64fc4cc8, State: Initialized, Role: FOLLOWER
I20260807 22:49:54.050246 11246 consensus_queue.cc:260] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } }
I20260807 22:49:54.050453 11246 raft_consensus.cc:388] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:54.050544 11246 raft_consensus.cc:482] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:54.050659 11246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:54.052937 11246 raft_consensus.cc:504] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } }
I20260807 22:49:54.053483 11246 leader_election.cc:302] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a499f5b7a64f478992aa6eaf64fc4cc8; no voters:
I20260807 22:49:54.054023 11246 leader_election.cc:288] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:54.054169 11251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:54.054868 11251 raft_consensus.cc:686] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [term 1 LEADER]: Becoming Leader. State: Replica: a499f5b7a64f478992aa6eaf64fc4cc8, State: Running, Role: LEADER
I20260807 22:49:54.055483 11251 consensus_queue.cc:237] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } }
I20260807 22:49:54.056093 11246 sys_catalog.cc:564] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:54.058485 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } } }
I20260807 22:49:54.058887 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:54.058604 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a499f5b7a64f478992aa6eaf64fc4cc8. Latest consensus state: current_term: 1 leader_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a499f5b7a64f478992aa6eaf64fc4cc8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39961 } } }
I20260807 22:49:54.059230 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:54.061506 11259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:54.065806 11259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:54.075685 11259 catalog_manager.cc:1347] Generated new cluster ID: 63eae8421926418bb1c86143d1039378
I20260807 22:49:54.075820 11259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:54.091542 11259 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:54.092825 11259 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:54.098850 11259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Generated new TSK 0
I20260807 22:49:54.099758 11259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:49:54.147104 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39961
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:54.475886 11272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:54.476208 11272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:54.476392 11272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:54.483491 11272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:54.483635 11272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:54.483798 11272 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:49:54.483942 11272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:49:54.494122 11272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:39961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:54.495499 11272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:54.496874 11272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:54.510082 11279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:54.510228 11278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:54.510089 11281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:54.510492 11272 server_base.cc:1034] running on GCE node
I20260807 22:49:54.511368 11272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:54.512354 11272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:54.513576 11272 hybrid_clock.cc:648] HybridClock initialized: now 1786142994513518 us; error 31 us; skew 500 ppm
I20260807 22:49:54.513885 11272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:54 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142994, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:54.517099 11272 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:49:54.518486 11272 webserver.cc:460] Webserver started at http://127.9.194.193:44415/ using document root <none> and password file <none>
I20260807 22:49:54.519176 11272 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:54.519292 11272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:54.519692 11272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:54.522559 11272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "dbfa32571b4a41519ed42b9b278cd0ad"
format_stamp: "Formatted at 2026-08-07 22:49:54 on dist-test-slave-5k54"
server_key: "102541d07b2e31035c8a28dc1381b31d"
server_key_iv: "2d7c95eb5899b66026646fd7a91a9ee4"
server_key_version: "encryptionkey@0"
I20260807 22:49:54.523324 11272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "dbfa32571b4a41519ed42b9b278cd0ad"
format_stamp: "Formatted at 2026-08-07 22:49:54 on dist-test-slave-5k54"
server_key: "102541d07b2e31035c8a28dc1381b31d"
server_key_iv: "2d7c95eb5899b66026646fd7a91a9ee4"
server_key_version: "encryptionkey@0"
I20260807 22:49:54.528353 11272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:49:54.531701 11289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:54.532954 11272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:49:54.533131 11272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "dbfa32571b4a41519ed42b9b278cd0ad"
format_stamp: "Formatted at 2026-08-07 22:49:54 on dist-test-slave-5k54"
server_key: "102541d07b2e31035c8a28dc1381b31d"
server_key_iv: "2d7c95eb5899b66026646fd7a91a9ee4"
server_key_version: "encryptionkey@0"
I20260807 22:49:54.533344 11272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:54.553817 11272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:54.557895 11272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:54.558185 11272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:54.561107 11272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:54.561209 11272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:54.561343 11272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:54.561450 11272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:49:54 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142994, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:54.587879 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:54.562413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.193:43725)
0807 22:49:54.563597 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:54.563679 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:54.565838 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:54.567003 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:54.567018 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:54.568047 (+ 1029us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:54.569147 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:54.569193 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.570600 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.570626 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:54.573462 (+ 2836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:54.573478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.574566 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.574577 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:54.574980 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:54.575946 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:54.575989 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:54.579007 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:54.582649 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:54.582661 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:54.582684 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:54.583300 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:54.583710 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:54.583715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:54.583723 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:54.584050 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:54.584833 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:54.584841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:54.585389 (+ 548us) client_negotiation.cc:769] Sending connection context
0807 22:49:54.585707 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 22:49:54.586242 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":453,"threads_started":1}
I20260807 22:49:54.588212 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:54.561664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:43725 (local address 127.9.194.254:39961)
0807 22:49:54.564309 (+ 2645us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:54.564316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:54.564340 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:49:54.566221 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:54.566227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:54.566326 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:54.566467 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:54.569469 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.570404 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.573656 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.574250 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.574400 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:54.579235 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:54.579240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:54.579243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:54.579275 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:54.582507 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.583465 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.583470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.583474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.583594 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.584174 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.584179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.584182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.584575 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:54.584742 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:54.587633 (+ 2891us) server_negotiation.cc:299] Negotiation successful
0807 22:49:54.587867 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":2374,"threads_started":1}
I20260807 22:49:54.613373 11419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:43239 every 8 connection(s)
I20260807 22:49:54.613373 11272 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:43239
I20260807 22:49:54.614485 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:54.594271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.193:36057)
0807 22:49:54.595005 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:54.595021 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:54.595169 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:54.595527 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:54.595533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:54.596227 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:54.596750 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:54.596770 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.598108 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.598111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:54.599713 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:54.599729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.600829 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.600837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:54.600951 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:54.605770 (+ 4819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:54.605799 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:54.607045 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:54.611492 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:54.611496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:54.611499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:54.611869 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:54.612340 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:54.612347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:54.612353 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:54.612550 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:54.613316 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:54.613324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:54.613554 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 22:49:54.613692 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 22:49:54.614022 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":424,"threads_started":1}
I20260807 22:49:54.615129 11272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:49:54.615600 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:54.594252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:36057 (local address 127.9.194.254:39961)
0807 22:49:54.594434 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:54.594440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:54.595104 (+ 664us) server_negotiation.cc:407] Connection header received
0807 22:49:54.595295 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:54.595301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:54.595370 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:54.595482 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:54.596954 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.597896 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.599919 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:54.600376 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:54.600509 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:54.607329 (+ 6820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:54.607338 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:54.607347 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:54.607403 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:54.611333 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.611987 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.611993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.611999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.612190 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:54.612747 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:54.612752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:54.612757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:54.613162 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:54.614866 (+ 1704us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:54.614998 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 22:49:54.615251 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:49:54.616971 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11272
I20260807 22:49:54.617195 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:49:54.617952 9995 external_mini_cluster.cc:1428] Setting key 3a0f6bfa51041b2976a002f639ab9937
I20260807 22:49:54.617751 11420 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39961
I20260807 22:49:54.618873 11420 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:54.619746 11420 heartbeater.cc:511] Master 127.9.194.254:39961 requested a full tablet report, sending...
I20260807 22:49:54.621655 11197 ts_manager.cc:194] Registered new tserver with Master: dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193:43239)
I20260807 22:49:54.623051 11197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:43725
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:49:54.660929 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39961
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:54.985544 11424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:54.985870 11424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:54.986057 11424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:54.993522 11424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:54.993688 11424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:54.993842 11424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:49:54.993989 11424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:49:55.003752 11424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:39961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:55.005165 11424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:55.006537 11424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:55.023579 11430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:55.023603 11431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:55.024369 11433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:55.028997 11424 server_base.cc:1034] running on GCE node
I20260807 22:49:55.029611 11424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:55.030539 11424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:55.031776 11424 hybrid_clock.cc:648] HybridClock initialized: now 1786142995031708 us; error 46 us; skew 500 ppm
I20260807 22:49:55.032076 11424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:55.035334 11424 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:49:55.036857 11424 webserver.cc:460] Webserver started at http://127.9.194.194:36295/ using document root <none> and password file <none>
I20260807 22:49:55.037632 11424 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:55.037760 11424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:55.038196 11424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:55.041865 11424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "b85a7a74f1a74243959fabe22f949dd1"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3d5ee3447a1dc82b21493c10c4c93967"
server_key_iv: "644756195ff55bd20c4ab2e460ae8153"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.042661 11424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "b85a7a74f1a74243959fabe22f949dd1"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3d5ee3447a1dc82b21493c10c4c93967"
server_key_iv: "644756195ff55bd20c4ab2e460ae8153"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.047809 11424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 22:49:55.051328 11440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:55.052744 11424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:55.052950 11424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "b85a7a74f1a74243959fabe22f949dd1"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3d5ee3447a1dc82b21493c10c4c93967"
server_key_iv: "644756195ff55bd20c4ab2e460ae8153"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.053177 11424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:55.084386 11424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:55.089048 11424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:55.089326 11424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:55.092459 11424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:55.092566 11424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:55.092721 11424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:55.092789 11424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:55.119113 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.092926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:57175 (local address 127.9.194.254:39961)
0807 22:49:55.093136 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.093147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.094700 (+ 1553us) server_negotiation.cc:407] Connection header received
0807 22:49:55.096960 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.096966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.097069 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.097191 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:55.099604 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.100970 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.103889 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.104762 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.104947 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.110319 (+ 5372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:55.110329 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:55.110337 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:55.110397 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:55.114654 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.115905 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.115910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.115912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.116038 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.116740 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.116746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.116749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.117155 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:55.117338 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.118559 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.118844 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:49:55.121261 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.093624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.194:57175)
0807 22:49:55.094520 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.094563 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.096658 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.097490 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.097505 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.098393 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:55.099347 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.099394 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.101168 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.101176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.103729 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.103744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.105333 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.105343 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.105654 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.106641 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:55.106685 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:55.110014 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:55.114806 (+ 4792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.114820 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.114846 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.115726 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.116165 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.116173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.116180 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.116569 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.117323 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:55.117336 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:55.118002 (+ 666us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.118412 (+ 410us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.119195 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":383,"threads_started":1}
I20260807 22:49:55.143805 11424 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:46513
I20260807 22:49:55.143874 11566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:46513 every 8 connection(s)
I20260807 22:49:55.144694 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.128475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.194:39321)
0807 22:49:55.129572 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.129600 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.129845 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.130272 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.130277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.130776 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:55.131079 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.131088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.132470 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.132479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.134023 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.134043 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.135357 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.135363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.135640 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.136436 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:55.136462 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:55.137475 (+ 1013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:55.141754 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.141758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.141761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.142149 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.142631 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.142636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.142639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.142773 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.143586 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:55.143596 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:55.143841 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.144041 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.144295 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":564,"threads_started":1}
I20260807 22:49:55.144833 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.128164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:39321 (local address 127.9.194.254:39961)
0807 22:49:55.128358 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.128365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.129701 (+ 1336us) server_negotiation.cc:407] Connection header received
0807 22:49:55.130049 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.130055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.130133 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.130245 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:55.131231 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.132326 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.134228 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.135135 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.135301 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.137666 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:55.137675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:55.137683 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:55.137733 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:55.141639 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.142315 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.142324 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.142330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.142501 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.142887 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.142895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.142901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.143404 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:55.143605 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.144273 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.144498 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:49:55.145397 11424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:49:55.147847 11568 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39961
I20260807 22:49:55.148116 11568 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:55.148862 11568 heartbeater.cc:511] Master 127.9.194.254:39961 requested a full tablet report, sending...
I20260807 22:49:55.150036 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11424
I20260807 22:49:55.150170 11197 ts_manager.cc:194] Registered new tserver with Master: b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513)
I20260807 22:49:55.150254 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:49:55.150775 9995 external_mini_cluster.cc:1428] Setting key 1774c96e5037e2010b63163aeee3134d
I20260807 22:49:55.151063 11197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:57175
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:49:55.194067 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39961
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:55.521384 11573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:55.521674 11573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:55.521772 11573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:55.529124 11573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:55.529237 11573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:55.529330 11573 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:49:55.529410 11573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:49:55.539492 11573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:39961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:55.540863 11573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:55.542255 11573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:55.555477 11579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:55.555603 11582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:55.555606 11580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:55.556373 11573 server_base.cc:1034] running on GCE node
I20260807 22:49:55.557029 11573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:55.558090 11573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:55.559336 11573 hybrid_clock.cc:648] HybridClock initialized: now 1786142995559273 us; error 43 us; skew 500 ppm
I20260807 22:49:55.559679 11573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:55.563086 11573 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:49:55.564621 11573 webserver.cc:460] Webserver started at http://127.9.194.195:37273/ using document root <none> and password file <none>
I20260807 22:49:55.565326 11573 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:55.565416 11573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:55.565829 11573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:55.568895 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "06e5a2f0fc084270b07f1918ab3821ad"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3afe8b4cc7606e26ee6dcabe8ee8ee16"
server_key_iv: "a9ffc859eb1fe7d6afebc060b56b5cb3"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.569696 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "06e5a2f0fc084270b07f1918ab3821ad"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3afe8b4cc7606e26ee6dcabe8ee8ee16"
server_key_iv: "a9ffc859eb1fe7d6afebc060b56b5cb3"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.574864 11573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:49:55.578954 11589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:55.580466 11573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:55.580739 11573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "06e5a2f0fc084270b07f1918ab3821ad"
format_stamp: "Formatted at 2026-08-07 22:49:55 on dist-test-slave-5k54"
server_key: "3afe8b4cc7606e26ee6dcabe8ee8ee16"
server_key_iv: "a9ffc859eb1fe7d6afebc060b56b5cb3"
server_key_version: "encryptionkey@0"
I20260807 22:49:55.580946 11573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:55.599211 11573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:55.603760 11573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:55.603998 11573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:55.606842 11573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:55.606956 11573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:55.607105 11573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:55.607250 11573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:55.625896 11420 heartbeater.cc:503] Master 127.9.194.254:39961 was elected leader, sending a full tablet report...
I20260807 22:49:55.632128 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.607326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:55491 (local address 127.9.194.254:39961)
0807 22:49:55.607572 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.607582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.609102 (+ 1520us) server_negotiation.cc:407] Connection header received
0807 22:49:55.611292 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.611297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.611400 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.611541 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:55.613474 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.614715 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.617833 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.618268 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.618420 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.623918 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:55.623924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:55.623927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:55.623962 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:55.627918 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.629154 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.629159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.629163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.629347 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.629903 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.629908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.629912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.630211 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:55.630418 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.631497 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.631742 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:49:55.633824 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.608068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.195:55491)
0807 22:49:55.608934 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.608978 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.610965 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.611793 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.611834 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.612634 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:55.613291 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.613309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.614931 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.614962 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.617617 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.617634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.618517 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.618524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.619061 (+ 537us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.620000 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:55.620041 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:55.623601 (+ 3560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:55.628126 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.628138 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.628168 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.629019 (+ 851us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.629503 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.629508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.629512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.629763 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.630359 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:55.630376 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:55.630972 (+ 596us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.631351 (+ 379us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.632015 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":383,"threads_started":1}
I20260807 22:49:55.655735 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.641664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:38379 (local address 127.9.194.254:39961)
0807 22:49:55.641872 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.641881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.642507 (+ 626us) server_negotiation.cc:407] Connection header received
0807 22:49:55.642665 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.642673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.642779 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.642916 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:55.643962 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.644676 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.646883 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.647585 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.647722 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.649934 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:55.649942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:55.649950 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:55.649982 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:55.652725 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.653393 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.653401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.653407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.653593 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.654107 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.654115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.654120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.654478 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:55.654636 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.655078 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.655305 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:49:55.655946 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.641720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.9.194.195:38379)
0807 22:49:55.642390 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.642424 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.642541 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.642928 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.642931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.643360 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:55.643799 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.643809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.644829 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.644834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.646651 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.646677 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.647792 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.647801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.647969 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.648664 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:55.648684 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:55.649726 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:55.652864 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.652869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.652874 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.653264 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.653732 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:55.653740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:55.653750 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:55.653965 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:55.654606 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:55.654610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:55.654815 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.654982 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.655385 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":376,"threads_started":1}
I20260807 22:49:55.663640 11573 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39753
I20260807 22:49:55.663688 11716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39753 every 8 connection(s)
I20260807 22:49:55.664912 11573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:49:55.666988 11717 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39961
I20260807 22:49:55.667176 11717 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:55.667881 11717 heartbeater.cc:511] Master 127.9.194.254:39961 requested a full tablet report, sending...
I20260807 22:49:55.669303 11197 ts_manager.cc:194] Registered new tserver with Master: 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:55.670399 11197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:55491
I20260807 22:49:55.673110 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11573
I20260807 22:49:55.673316 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:49:55.673746 9995 external_mini_cluster.cc:1428] Setting key 10d4a166ed4a440cc447e094a4c2c43c
I20260807 22:49:55.676826 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:49:55 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:55.725697 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.712301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.9.194.254:39961)
0807 22:49:55.712570 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.712578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.712820 (+ 242us) server_negotiation.cc:407] Connection header received
0807 22:49:55.712983 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.712987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.713054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.713139 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:55.714470 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.715181 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.716437 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.716844 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.716947 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.720795 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:55.720801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:55.720804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:55.720838 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:55.723256 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.724010 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.724014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.724016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.724131 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:55.724502 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:55.724513 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:55.724515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:55.724799 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:55.724985 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.725246 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.725411 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 22:49:55.729733 11197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:49:55.736675 11197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:49:55.772747 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.748929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43239 (local address 127.0.0.1:33686)
0807 22:49:55.760153 (+ 11224us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.760217 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.760457 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.766017 (+ 5560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.766021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.766050 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:55.766482 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.766496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.768399 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.768404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.770350 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.770361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.771863 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.771870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.772036 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.772068 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.772156 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.772309 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10871,"mutex_wait_us":23,"thread_start_us":15143,"threads_started":1}
I20260807 22:49:55.772888 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.749899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.9.194.194:46513)
0807 22:49:55.763563 (+ 13664us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.763573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.763928 (+ 355us) server_negotiation.cc:407] Connection header received
0807 22:49:55.764151 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.764160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.764467 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.764679 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:55.765701 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.767110 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.769850 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.771278 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.771470 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.771702 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.772431 (+ 729us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.772601 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13434,"thread_start_us":13662,"threads_started":1}
I20260807 22:49:55.773523 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.749152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.9.194.193:43239)
0807 22:49:55.765414 (+ 16262us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.765426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.765454 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:49:55.765557 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.765567 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.765808 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.766003 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:55.766685 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.768254 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.770502 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.771531 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.771664 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.771884 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.773011 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.773178 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16029,"thread_start_us":15897,"threads_started":1}
I20260807 22:49:55.773973 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.749650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:46513 (local address 127.0.0.1:51556)
0807 22:49:55.763779 (+ 14129us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.763818 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.763965 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.764981 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.764990 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.765025 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:55.765449 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.765464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.767251 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.767257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.769656 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.769682 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.771549 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.771557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.771742 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.771800 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.771937 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.772121 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13997,"mutex_wait_us":52,"thread_start_us":9946,"threads_started":1}
I20260807 22:49:55.774510 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.750052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39753 (local address 127.0.0.1:46474)
0807 22:49:55.764678 (+ 14626us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:55.764701 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:55.764826 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:55.767027 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:55.767032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:55.767060 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:55.767664 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.767676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.769265 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.769270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.771307 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:55.771322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.773470 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.773476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:55.773655 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.773686 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:49:55.773779 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 22:49:55.773924 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14506,"mutex_wait_us":47,"thread_start_us":14345,"threads_started":1}
I20260807 22:49:55.774653 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:55.750700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.9.194.195:39753)
0807 22:49:55.765907 (+ 15207us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:55.765913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:55.766390 (+ 477us) server_negotiation.cc:407] Connection header received
0807 22:49:55.766497 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:55.766505 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:55.766801 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:55.767360 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:55.767906 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.769116 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.771507 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:55.772997 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:55.773119 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:55.773340 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:55.774071 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 22:49:55.774278 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15062,"thread_start_us":14835,"threads_started":1}
I20260807 22:49:55.777822 11643 tablet_service.cc:1467] Processing CreateTablet for tablet a7faf27ae9a145419b577fb09af00d9d (DEFAULT_TABLE table=test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:55.777984 11347 tablet_service.cc:1467] Processing CreateTablet for tablet a7faf27ae9a145419b577fb09af00d9d (DEFAULT_TABLE table=test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:55.777972 11492 tablet_service.cc:1467] Processing CreateTablet for tablet a7faf27ae9a145419b577fb09af00d9d (DEFAULT_TABLE table=test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:55.779146 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7faf27ae9a145419b577fb09af00d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:55.779287 11492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7faf27ae9a145419b577fb09af00d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:55.779646 11347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7faf27ae9a145419b577fb09af00d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:55.791066 11737 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Bootstrap starting.
I20260807 22:49:55.793589 11739 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Bootstrap starting.
I20260807 22:49:55.795226 11737 tablet_bootstrap.cc:654] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:55.796094 11738 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Bootstrap starting.
I20260807 22:49:55.796581 11737 log.cc:824] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:55.797181 11739 tablet_bootstrap.cc:654] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:55.798628 11739 log.cc:824] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:55.799571 11738 tablet_bootstrap.cc:654] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:55.800858 11738 log.cc:824] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:55.801515 11737 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: No bootstrap required, opened a new log
I20260807 22:49:55.801826 11739 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: No bootstrap required, opened a new log
I20260807 22:49:55.801815 11737 ts_tablet_manager.cc:1397] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 22:49:55.802094 11739 ts_tablet_manager.cc:1397] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 22:49:55.803872 11738 tablet_bootstrap.cc:492] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: No bootstrap required, opened a new log
I20260807 22:49:55.804142 11738 ts_tablet_manager.cc:1397] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 22:49:55.809167 11737 raft_consensus.cc:348] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.809587 11737 raft_consensus.cc:374] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:55.809717 11737 raft_consensus.cc:729] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e5a2f0fc084270b07f1918ab3821ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:55.810343 11737 consensus_queue.cc:260] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.811529 11717 heartbeater.cc:503] Master 127.9.194.254:39961 was elected leader, sending a full tablet report...
I20260807 22:49:55.812306 11737 ts_tablet_manager.cc:1428] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260807 22:49:55.813071 11739 raft_consensus.cc:348] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.813629 11739 raft_consensus.cc:374] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:55.813774 11739 raft_consensus.cc:729] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfa32571b4a41519ed42b9b278cd0ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:55.814412 11739 consensus_queue.cc:260] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.814546 11738 raft_consensus.cc:348] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.814965 11738 raft_consensus.cc:374] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:55.815106 11738 raft_consensus.cc:729] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85a7a74f1a74243959fabe22f949dd1, State: Initialized, Role: FOLLOWER
I20260807 22:49:55.815884 11738 consensus_queue.cc:260] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:55.817443 11568 heartbeater.cc:503] Master 127.9.194.254:39961 was elected leader, sending a full tablet report...
I20260807 22:49:55.817720 11739 ts_tablet_manager.cc:1428] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260807 22:49:55.819139 11738 ts_tablet_manager.cc:1428] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260807 22:49:55.868016 11421 tablet.cc:2369] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:49:55.897703 11569 tablet.cc:2369] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:49:55.916904 11718 tablet.cc:2369] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:56.007831 11744 raft_consensus.cc:482] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:49:56.008128 11744 raft_consensus.cc:504] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.010038 11744 leader_election.cc:288] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.019392 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.010791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:60281 (local address 127.9.194.195:39753)
0807 22:49:56.011091 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.011100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.011808 (+ 708us) server_negotiation.cc:407] Connection header received
0807 22:49:56.012091 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.012096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.012172 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.012279 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:56.013044 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.013931 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.017387 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.018243 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.018461 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.018546 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.018987 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.019134 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 22:49:56.019543 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.010304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39753 (local address 127.9.194.193:60281)
0807 22:49:56.011677 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.011692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.011919 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.012298 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.012303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.012329 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:56.012869 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.012882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.014091 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.014097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.017161 (+ 3064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.017178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.018487 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.018494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.018714 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.018767 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.018927 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.019074 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"mutex_wait_us":351,"thread_start_us":546,"threads_started":1}
I20260807 22:49:56.019717 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.010489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:50231 (local address 127.9.194.194:46513)
0807 22:49:56.010744 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.010753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.011492 (+ 739us) server_negotiation.cc:407] Connection header received
0807 22:49:56.011847 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.011853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.011952 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.012053 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:56.012887 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.014101 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.017333 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.018596 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.018771 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.018841 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.019252 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.019405 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:49:56.020421 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.010304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:46513 (local address 127.9.194.193:50231)
0807 22:49:56.011343 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.011359 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.011632 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.012099 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.012104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.012131 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:56.012689 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.012706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.014239 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.014246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.017161 (+ 2915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.017178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.018843 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.018854 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.019045 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.019077 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.019934 (+ 857us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.020095 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":109,"thread_start_us":1042,"threads_started":1}
I20260807 22:49:56.020710 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7faf27ae9a145419b577fb09af00d9d" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad" is_pre_election: true
I20260807 22:49:56.021330 11665 raft_consensus.cc:2454] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 0.
I20260807 22:49:56.021481 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7faf27ae9a145419b577fb09af00d9d" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85a7a74f1a74243959fabe22f949dd1" is_pre_election: true
I20260807 22:49:56.022023 11513 raft_consensus.cc:2454] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 0.
I20260807 22:49:56.022137 11290 leader_election.cc:302] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e5a2f0fc084270b07f1918ab3821ad, dbfa32571b4a41519ed42b9b278cd0ad; no voters:
I20260807 22:49:56.022656 11744 raft_consensus.cc:2781] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:49:56.022833 11744 raft_consensus.cc:482] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:49:56.022938 11744 raft_consensus.cc:3037] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.026270 11744 raft_consensus.cc:504] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.026929 11744 leader_election.cc:288] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 election: Requested vote from peers b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.027470 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7faf27ae9a145419b577fb09af00d9d" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad"
I20260807 22:49:56.027673 11665 raft_consensus.cc:3037] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.030818 11665 raft_consensus.cc:2454] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 1.
I20260807 22:49:56.031227 11290 leader_election.cc:302] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e5a2f0fc084270b07f1918ab3821ad, dbfa32571b4a41519ed42b9b278cd0ad; no voters:
I20260807 22:49:56.031206 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7faf27ae9a145419b577fb09af00d9d" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85a7a74f1a74243959fabe22f949dd1"
I20260807 22:49:56.031493 11513 raft_consensus.cc:3037] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.031540 11744 raft_consensus.cc:2781] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:56.033994 11513 raft_consensus.cc:2454] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 1.
I20260807 22:49:56.034418 11744 raft_consensus.cc:686] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 1 LEADER]: Becoming Leader. State: Replica: dbfa32571b4a41519ed42b9b278cd0ad, State: Running, Role: LEADER
I20260807 22:49:56.035044 11744 consensus_queue.cc:237] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.040414 11197 catalog_manager.cc:5495] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193). New cstate: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } health_report { overall_health: UNKNOWN } } }
I20260807 22:49:56.137712 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.131862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.9.194.193:43239)
0807 22:49:56.132113 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.132120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.132255 (+ 135us) server_negotiation.cc:407] Connection header received
0807 22:49:56.132620 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.132624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.132722 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.132845 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:56.133178 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.133874 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.135332 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.135807 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.135900 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.136391 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.137083 (+ 692us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.137189 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.137356 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.137461 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:49:56.147234 11665 raft_consensus.cc:1261] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Refusing update from remote peer dbfa32571b4a41519ed42b9b278cd0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:56.148437 11748 consensus_queue.cc:1035] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.149076 11513 raft_consensus.cc:1261] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Refusing update from remote peer dbfa32571b4a41519ed42b9b278cd0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:56.150012 11748 consensus_queue.cc:1035] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.184677 11196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:49:56.185137 11196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:49:56.189536 11196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Sending DeleteTablet for 3 replicas of tablet a7faf27ae9a145419b577fb09af00d9d
I20260807 22:49:56.190877 11347 tablet_service.cc:1514] Processing DeleteTablet for tablet a7faf27ae9a145419b577fb09af00d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:33686
I20260807 22:49:56.190912 11643 tablet_service.cc:1514] Processing DeleteTablet for tablet a7faf27ae9a145419b577fb09af00d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:46474
I20260807 22:49:56.190936 11492 tablet_service.cc:1514] Processing DeleteTablet for tablet a7faf27ae9a145419b577fb09af00d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51556
I20260807 22:49:56.195066 11197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:49:56.195948 11197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:49:56.199899 11762 tablet_replica.cc:330] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: stopping tablet replica
I20260807 22:49:56.200729 11762 raft_consensus.cc:2229] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:49:56.201447 11761 tablet_replica.cc:330] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: stopping tablet replica
I20260807 22:49:56.202126 11762 raft_consensus.cc:2258] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.202710 11761 raft_consensus.cc:2229] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:49:56.202960 11763 tablet_replica.cc:330] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: stopping tablet replica
I20260807 22:49:56.203478 11761 raft_consensus.cc:2258] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.204823 11643 tablet_service.cc:1467] Processing CreateTablet for tablet ffb215de628d4dec936b6ed74e354f68 (DEFAULT_TABLE table=test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:56.204972 11492 tablet_service.cc:1467] Processing CreateTablet for tablet ffb215de628d4dec936b6ed74e354f68 (DEFAULT_TABLE table=test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:56.205307 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb215de628d4dec936b6ed74e354f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.205390 11492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb215de628d4dec936b6ed74e354f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.206141 11761 ts_tablet_manager.cc:1905] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.208808 11763 raft_consensus.cc:2229] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:49:56.207808 11347 tablet_service.cc:1467] Processing CreateTablet for tablet ffb215de628d4dec936b6ed74e354f68 (DEFAULT_TABLE table=test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:49:56.209949 11347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb215de628d4dec936b6ed74e354f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.211824 11762 ts_tablet_manager.cc:1905] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.212495 11738 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Bootstrap starting.
I20260807 22:49:56.214182 11738 tablet_bootstrap.cc:654] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.214395 11763 raft_consensus.cc:2258] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.219069 11739 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Bootstrap starting.
I20260807 22:49:56.221418 11739 tablet_bootstrap.cc:654] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.223007 11761 ts_tablet_manager.cc:1918] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:49:56.223394 11761 log.cc:1196] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/a7faf27ae9a145419b577fb09af00d9d
I20260807 22:49:56.224180 11761 ts_tablet_manager.cc:1939] T a7faf27ae9a145419b577fb09af00d9d P b85a7a74f1a74243959fabe22f949dd1: Deleting consensus metadata
I20260807 22:49:56.226337 11184 catalog_manager.cc:4841] TS b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513): tablet a7faf27ae9a145419b577fb09af00d9d (table test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]) successfully deleted
I20260807 22:49:56.228075 11763 ts_tablet_manager.cc:1905] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.228199 11738 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: No bootstrap required, opened a new log
I20260807 22:49:56.228456 11738 ts_tablet_manager.cc:1397] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260807 22:49:56.229401 11738 raft_consensus.cc:348] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.229718 11738 raft_consensus.cc:374] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.229851 11738 raft_consensus.cc:729] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85a7a74f1a74243959fabe22f949dd1, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.230221 11738 consensus_queue.cc:260] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.230904 11738 ts_tablet_manager.cc:1428] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:49:56.233278 11737 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Bootstrap starting.
I20260807 22:49:56.234905 11737 tablet_bootstrap.cc:654] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.237711 11739 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: No bootstrap required, opened a new log
I20260807 22:49:56.237895 11739 ts_tablet_manager.cc:1397] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20260807 22:49:56.238127 11762 ts_tablet_manager.cc:1918] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:49:56.238281 11762 log.cc:1196] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/a7faf27ae9a145419b577fb09af00d9d
I20260807 22:49:56.238962 11762 ts_tablet_manager.cc:1939] T a7faf27ae9a145419b577fb09af00d9d P 06e5a2f0fc084270b07f1918ab3821ad: Deleting consensus metadata
I20260807 22:49:56.239006 11739 raft_consensus.cc:348] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.239267 11739 raft_consensus.cc:374] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.239393 11739 raft_consensus.cc:729] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfa32571b4a41519ed42b9b278cd0ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.239811 11739 consensus_queue.cc:260] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.239966 11737 tablet_bootstrap.cc:492] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: No bootstrap required, opened a new log
I20260807 22:49:56.240108 11737 ts_tablet_manager.cc:1397] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 22:49:56.240922 11737 raft_consensus.cc:348] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.241160 11737 raft_consensus.cc:374] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.241173 11181 catalog_manager.cc:4841] TS 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753): tablet a7faf27ae9a145419b577fb09af00d9d (table test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]) successfully deleted
I20260807 22:49:56.241277 11737 raft_consensus.cc:729] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e5a2f0fc084270b07f1918ab3821ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.241571 11737 consensus_queue.cc:260] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.241864 11739 ts_tablet_manager.cc:1428] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 22:49:56.243464 11737 ts_tablet_manager.cc:1428] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:56.246266 11763 ts_tablet_manager.cc:1918] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:49:56.246470 11763 log.cc:1196] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/a7faf27ae9a145419b577fb09af00d9d
I20260807 22:49:56.247154 11763 ts_tablet_manager.cc:1939] T a7faf27ae9a145419b577fb09af00d9d P dbfa32571b4a41519ed42b9b278cd0ad: Deleting consensus metadata
I20260807 22:49:56.248705 11182 catalog_manager.cc:4841] TS dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193:43239): tablet a7faf27ae9a145419b577fb09af00d9d (table test-table [id=28cd2b1e527843a590a36fb0ccf0eb0b]) successfully deleted
I20260807 22:49:56.287711 11745 raft_consensus.cc:482] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:49:56.287976 11745 raft_consensus.cc:504] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.289610 11745 leader_election.cc:288] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193:43239), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.297957 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.290043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43239 (local address 127.9.194.194:38125)
0807 22:49:56.291323 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.291344 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.291592 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.292185 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.292192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.292220 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:56.292664 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.292682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.294195 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.294199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.295773 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.295785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.297125 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.297132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.297317 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.297355 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.297457 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.297594 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"thread_start_us":974,"threads_started":1}
I20260807 22:49:56.297983 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.290253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:38125 (local address 127.9.194.193:43239)
0807 22:49:56.290570 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.290577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.291451 (+ 874us) server_negotiation.cc:407] Connection header received
0807 22:49:56.291789 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.291796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.291894 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.291960 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:56.292830 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.294041 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.295943 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.296899 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.297069 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.297145 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.297510 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.297653 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260807 22:49:56.299067 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.290542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:45519 (local address 127.9.194.195:39753)
0807 22:49:56.290819 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.290828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.292214 (+ 1386us) server_negotiation.cc:407] Connection header received
0807 22:49:56.292502 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.292510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.292601 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.292667 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:49:56.293342 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.294081 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.296676 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.297779 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.297923 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.298020 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.298644 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.298786 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260807 22:49:56.299119 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.290048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39753 (local address 127.9.194.194:45519)
0807 22:49:56.292122 (+ 2074us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.292140 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.292322 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.292885 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.292888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.292907 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:49:56.293172 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.293178 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.294242 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.294248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.296503 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.296519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.298234 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.298243 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.298445 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.298488 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.298595 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.298720 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1945,"mutex_wait_us":112,"thread_start_us":2325,"threads_started":1}
I20260807 22:49:56.299129 11372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb215de628d4dec936b6ed74e354f68" candidate_uuid: "b85a7a74f1a74243959fabe22f949dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" is_pre_election: true
I20260807 22:49:56.299675 11372 raft_consensus.cc:2454] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85a7a74f1a74243959fabe22f949dd1 in term 0.
I20260807 22:49:56.299860 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb215de628d4dec936b6ed74e354f68" candidate_uuid: "b85a7a74f1a74243959fabe22f949dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad" is_pre_election: true
I20260807 22:49:56.300112 11665 raft_consensus.cc:2454] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85a7a74f1a74243959fabe22f949dd1 in term 0.
I20260807 22:49:56.300418 11443 leader_election.cc:302] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b85a7a74f1a74243959fabe22f949dd1, dbfa32571b4a41519ed42b9b278cd0ad; no voters:
I20260807 22:49:56.300920 11745 raft_consensus.cc:2781] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:49:56.301052 11745 raft_consensus.cc:482] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:49:56.301146 11745 raft_consensus.cc:3037] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.304342 11745 raft_consensus.cc:504] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.305090 11745 leader_election.cc:288] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [CANDIDATE]: Term 1 election: Requested vote from peers dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193:43239), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.305516 11372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb215de628d4dec936b6ed74e354f68" candidate_uuid: "b85a7a74f1a74243959fabe22f949dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbfa32571b4a41519ed42b9b278cd0ad"
I20260807 22:49:56.305543 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb215de628d4dec936b6ed74e354f68" candidate_uuid: "b85a7a74f1a74243959fabe22f949dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad"
I20260807 22:49:56.305716 11372 raft_consensus.cc:3037] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.305737 11665 raft_consensus.cc:3037] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.307948 11665 raft_consensus.cc:2454] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85a7a74f1a74243959fabe22f949dd1 in term 1.
I20260807 22:49:56.308351 11442 leader_election.cc:302] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e5a2f0fc084270b07f1918ab3821ad, b85a7a74f1a74243959fabe22f949dd1; no voters:
I20260807 22:49:56.308650 11745 raft_consensus.cc:2781] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:56.309131 11372 raft_consensus.cc:2454] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85a7a74f1a74243959fabe22f949dd1 in term 1.
I20260807 22:49:56.309181 11745 raft_consensus.cc:686] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 1 LEADER]: Becoming Leader. State: Replica: b85a7a74f1a74243959fabe22f949dd1, State: Running, Role: LEADER
I20260807 22:49:56.309686 11745 consensus_queue.cc:237] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.313225 11196 catalog_manager.cc:5495] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "b85a7a74f1a74243959fabe22f949dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:49:56 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142993, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:56.374022 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.358112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39961 (local address 127.0.0.1:49708)
0807 22:49:56.359002 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.359019 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.359141 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.359575 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.359579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.360224 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:56.360428 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.360437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.361966 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.361970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.363077 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.363088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.364071 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.364075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.364205 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.364927 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:56.364945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:56.367495 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:56.371410 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:56.371474 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:56.371485 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:56.371951 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:56.372377 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:56.372381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:56.372383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:56.372553 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:56.373176 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:56.373182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:56.373352 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.373496 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.373728 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":598,"threads_started":1}
I20260807 22:49:56.374346 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.358112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.9.194.254:39961)
0807 22:49:56.358783 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.358790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.359110 (+ 320us) server_negotiation.cc:407] Connection header received
0807 22:49:56.359256 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.359262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.359343 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.359496 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:56.360590 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.361809 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.363207 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.363912 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.364051 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.367684 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:56.367691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:56.367696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:56.367735 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:56.371249 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.372066 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.372072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.372076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.372227 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.372666 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.372671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.372675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.373043 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:56.373191 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.373608 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.373894 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":508,"thread_start_us":334,"threads_started":1}
I20260807 22:49:56.379899 11196 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:49708:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:49:56.381088 11196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:49:56.388053 11347 tablet_service.cc:1467] Processing CreateTablet for tablet e69cf9b1b51c4db3bd0073ffe18eeab3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc61247afe29481a9ae25b92f766cd6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:49:56.388062 11492 tablet_service.cc:1467] Processing CreateTablet for tablet e69cf9b1b51c4db3bd0073ffe18eeab3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc61247afe29481a9ae25b92f766cd6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:49:56.388370 11643 tablet_service.cc:1467] Processing CreateTablet for tablet e69cf9b1b51c4db3bd0073ffe18eeab3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc61247afe29481a9ae25b92f766cd6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:49:56.388655 11492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69cf9b1b51c4db3bd0073ffe18eeab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.388664 11347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69cf9b1b51c4db3bd0073ffe18eeab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.388902 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69cf9b1b51c4db3bd0073ffe18eeab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:56.396420 11739 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: Bootstrap starting.
I20260807 22:49:56.396487 11738 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: Bootstrap starting.
I20260807 22:49:56.398245 11738 tablet_bootstrap.cc:654] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.398250 11739 tablet_bootstrap.cc:654] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.400372 11737 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: Bootstrap starting.
I20260807 22:49:56.402122 11737 tablet_bootstrap.cc:654] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:56.408104 11737 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: No bootstrap required, opened a new log
I20260807 22:49:56.408409 11737 ts_tablet_manager.cc:1397] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 22:49:56.409700 11739 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: No bootstrap required, opened a new log
I20260807 22:49:56.409564 11737 raft_consensus.cc:348] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.409899 11739 ts_tablet_manager.cc:1397] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260807 22:49:56.409924 11737 raft_consensus.cc:374] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.410019 11737 raft_consensus.cc:729] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e5a2f0fc084270b07f1918ab3821ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.410326 11737 consensus_queue.cc:260] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.410692 11739 raft_consensus.cc:348] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.410883 11743 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.411137 11739 raft_consensus.cc:374] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.411268 11743 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.411278 11739 raft_consensus.cc:729] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfa32571b4a41519ed42b9b278cd0ad, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.411559 11739 consensus_queue.cc:260] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.412045 11744 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.412321 11744 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.413182 11737 ts_tablet_manager.cc:1428] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 22:49:56.413976 11739 ts_tablet_manager.cc:1428] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 22:49:56.417042 11738 tablet_bootstrap.cc:492] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: No bootstrap required, opened a new log
I20260807 22:49:56.417234 11738 ts_tablet_manager.cc:1397] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.011s
I20260807 22:49:56.417991 11738 raft_consensus.cc:348] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.418241 11738 raft_consensus.cc:374] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:56.418322 11738 raft_consensus.cc:729] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85a7a74f1a74243959fabe22f949dd1, State: Initialized, Role: FOLLOWER
I20260807 22:49:56.418573 11738 consensus_queue.cc:260] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.419064 11745 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.419231 11738 ts_tablet_manager.cc:1428] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 22:49:56.419370 11745 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.638365 11744 raft_consensus.cc:482] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:49:56.638664 11744 raft_consensus.cc:504] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.639532 11744 leader_election.cc:288] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.640300 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69cf9b1b51c4db3bd0073ffe18eeab3" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85a7a74f1a74243959fabe22f949dd1" is_pre_election: true
I20260807 22:49:56.640568 11513 raft_consensus.cc:2454] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 0.
I20260807 22:49:56.640991 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69cf9b1b51c4db3bd0073ffe18eeab3" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad" is_pre_election: true
I20260807 22:49:56.641207 11294 leader_election.cc:302] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b85a7a74f1a74243959fabe22f949dd1, dbfa32571b4a41519ed42b9b278cd0ad; no voters:
I20260807 22:49:56.641295 11665 raft_consensus.cc:2454] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 0.
I20260807 22:49:56.641671 11744 raft_consensus.cc:2781] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:49:56.641779 11744 raft_consensus.cc:482] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:49:56.641918 11744 raft_consensus.cc:3037] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.644307 11744 raft_consensus.cc:504] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.645061 11744 leader_election.cc:288] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 election: Requested vote from peers b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513), 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753)
I20260807 22:49:56.645362 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69cf9b1b51c4db3bd0073ffe18eeab3" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85a7a74f1a74243959fabe22f949dd1"
I20260807 22:49:56.645586 11513 raft_consensus.cc:3037] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.645613 11665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69cf9b1b51c4db3bd0073ffe18eeab3" candidate_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e5a2f0fc084270b07f1918ab3821ad"
I20260807 22:49:56.645828 11665 raft_consensus.cc:3037] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:56.648751 11665 raft_consensus.cc:2454] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 1.
I20260807 22:49:56.648767 11513 raft_consensus.cc:2454] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfa32571b4a41519ed42b9b278cd0ad in term 1.
I20260807 22:49:56.649247 11294 leader_election.cc:302] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06e5a2f0fc084270b07f1918ab3821ad, b85a7a74f1a74243959fabe22f949dd1, dbfa32571b4a41519ed42b9b278cd0ad; no voters:
I20260807 22:49:56.649628 11744 raft_consensus.cc:2781] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:56.649840 11744 raft_consensus.cc:686] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 LEADER]: Becoming Leader. State: Replica: dbfa32571b4a41519ed42b9b278cd0ad, State: Running, Role: LEADER
I20260807 22:49:56.650120 11744 consensus_queue.cc:237] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } }
I20260807 22:49:56.652086 11748 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: TxnStatusTablet state changed. Reason: New leader dbfa32571b4a41519ed42b9b278cd0ad. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.652390 11748 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:49:56.653165 11782 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:49:56.653436 11782 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:49:56.653668 11197 catalog_manager.cc:5495] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193). New cstate: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } health_report { overall_health: UNKNOWN } } }
I20260807 22:49:56.712471 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.706624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43239 (local address 127.0.0.1:33714)
0807 22:49:56.706913 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.706930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.707052 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.707547 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.707550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.707561 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:49:56.707710 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.707716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.709196 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.709203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.710357 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.710369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.711334 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.711339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.711582 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.711638 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.712041 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.712056 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.712117 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.712230 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 22:49:56.712566 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.706621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.9.194.193:43239)
0807 22:49:56.706831 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.706838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.707008 (+ 170us) server_negotiation.cc:407] Connection header received
0807 22:49:56.707221 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.707227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.707348 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.707492 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:56.707840 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.709048 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.710500 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.711179 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.711302 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.711802 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.711940 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.712035 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.712167 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.712247 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:49:56.717263 11513 raft_consensus.cc:1261] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Refusing update from remote peer dbfa32571b4a41519ed42b9b278cd0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:56.717370 11665 raft_consensus.cc:1261] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Refusing update from remote peer dbfa32571b4a41519ed42b9b278cd0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:56.717971 11748 consensus_queue.cc:1035] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.718271 11744 consensus_queue.cc:1035] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.726531 11743 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: TxnStatusTablet state changed. Reason: New leader dbfa32571b4a41519ed42b9b278cd0ad. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.726825 11743 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.728319 11784 mvcc.cc:205] Tried to move back new op lower bound from 7316041714547863552 to 7316041714281803776. Current Snapshot: MvccSnapshot[applied={T|T < 7316041714547863552}]
I20260807 22:49:56.728367 11745 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: TxnStatusTablet state changed. Reason: New leader dbfa32571b4a41519ed42b9b278cd0ad. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.728641 11745 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.728946 11786 mvcc.cc:205] Tried to move back new op lower bound from 7316041714547863552 to 7316041714281803776. Current Snapshot: MvccSnapshot[applied={T|T < 7316041714547863552}]
I20260807 22:49:56.730513 11744 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.730837 11744 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:49:56.731110 11744 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.731276 11744 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P dbfa32571b4a41519ed42b9b278cd0ad: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:49:56.733055 11743 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.733320 11743 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P 06e5a2f0fc084270b07f1918ab3821ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.737404 11745 tablet_replica.cc:439] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 } } peers { permanent_uuid: "b85a7a74f1a74243959fabe22f949dd1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 46513 } } peers { permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 } } }
I20260807 22:49:56.737671 11745 tablet_replica.cc:442] T e69cf9b1b51c4db3bd0073ffe18eeab3 P b85a7a74f1a74243959fabe22f949dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:49:56.747956 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.740339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.9.194.194:46513)
0807 22:49:56.741828 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.741832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.741846 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:49:56.741928 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.741933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.742021 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.742150 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:56.742738 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.743548 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.745620 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.745996 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.746087 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.746630 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.747238 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.747346 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.747527 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.747607 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1404,"thread_start_us":1072,"threads_started":1}
I20260807 22:49:56.767786 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.759523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:56619 (local address 127.9.194.193:43239)
0807 22:49:56.759767 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.759774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.762088 (+ 2314us) server_negotiation.cc:407] Connection header received
0807 22:49:56.762239 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.762245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.762348 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.762455 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:56.762994 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.764253 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.765683 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.766445 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.766570 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.766956 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.767116 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.767203 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.767357 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.767483 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:49:56.767820 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.759382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43239 (local address 127.9.194.194:56619)
0807 22:49:56.761982 (+ 2600us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.761998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.762121 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.762471 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.762475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.762484 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:49:56.762805 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.762813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.764411 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.764415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.765535 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.765548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.766611 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.766616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.766729 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.766787 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.767225 (+ 438us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.767234 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.767293 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.767402 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2426,"thread_start_us":2279,"threads_started":1}
I20260807 22:49:56.773478 11745 consensus_queue.cc:1035] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbfa32571b4a41519ed42b9b278cd0ad" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.780601 11665 raft_consensus.cc:1261] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Refusing update from remote peer b85a7a74f1a74243959fabe22f949dd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:56.781419 11745 consensus_queue.cc:1035] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e5a2f0fc084270b07f1918ab3821ad" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:56.826308 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.816787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:47191 (local address 127.9.194.194:46513)
0807 22:49:56.816962 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.816969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.820064 (+ 3095us) server_negotiation.cc:407] Connection header received
0807 22:49:56.820293 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.820298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.820378 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.820476 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:56.821111 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.822235 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.823845 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.824570 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.824703 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.825253 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.825411 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.825516 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.825867 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.825955 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:49:56.826457 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.816668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:46513 (local address 127.9.194.193:47191)
0807 22:49:56.819952 (+ 3284us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:56.819971 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:56.820145 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:56.820497 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:56.820506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:56.820519 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:49:56.820935 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.820949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.822357 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.822363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.823713 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:56.823725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.824766 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.824774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:56.825004 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.825077 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:49:56.825707 (+ 630us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:49:56.825725 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:49:56.825817 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:49:56.825967 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3076,"thread_start_us":2946,"threads_started":1}
I20260807 22:49:56.880698 11197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:49:56.881026 11197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:49:56.884552 11197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a499f5b7a64f478992aa6eaf64fc4cc8: Sending DeleteTablet for 3 replicas of tablet ffb215de628d4dec936b6ed74e354f68
I20260807 22:49:56.885581 11347 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb215de628d4dec936b6ed74e354f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:33686
I20260807 22:49:56.885907 11643 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb215de628d4dec936b6ed74e354f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:46474
I20260807 22:49:56.886713 11492 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb215de628d4dec936b6ed74e354f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:49:56 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51556
I20260807 22:49:56.901436 11808 tablet_replica.cc:330] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: stopping tablet replica
I20260807 22:49:56.902896 11808 raft_consensus.cc:2229] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:49:56.903842 11809 tablet_replica.cc:330] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: stopping tablet replica
I20260807 22:49:56.904081 11809 raft_consensus.cc:2229] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:49:56.904332 11808 raft_consensus.cc:2258] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.904937 11809 raft_consensus.cc:2258] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.905941 11807 tablet_replica.cc:330] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: stopping tablet replica
I20260807 22:49:56.906198 11807 raft_consensus.cc:2229] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:49:56.906287 11809 ts_tablet_manager.cc:1905] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.906581 11807 raft_consensus.cc:2258] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:49:56.907826 11807 ts_tablet_manager.cc:1905] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.915416 11808 ts_tablet_manager.cc:1905] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:49:56.920972 11807 ts_tablet_manager.cc:1918] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:49:56.924494 11807 log.cc:1196] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/ffb215de628d4dec936b6ed74e354f68
I20260807 22:49:56.925122 11807 ts_tablet_manager.cc:1939] T ffb215de628d4dec936b6ed74e354f68 P dbfa32571b4a41519ed42b9b278cd0ad: Deleting consensus metadata
I20260807 22:49:56.927122 11809 ts_tablet_manager.cc:1918] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:49:56.928297 11182 catalog_manager.cc:4841] TS dbfa32571b4a41519ed42b9b278cd0ad (127.9.194.193:43239): tablet ffb215de628d4dec936b6ed74e354f68 (table test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]) successfully deleted
I20260807 22:49:56.929857 11808 ts_tablet_manager.cc:1918] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:49:56.930107 11809 log.cc:1196] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/ffb215de628d4dec936b6ed74e354f68
I20260807 22:49:56.930728 11809 ts_tablet_manager.cc:1939] T ffb215de628d4dec936b6ed74e354f68 P b85a7a74f1a74243959fabe22f949dd1: Deleting consensus metadata
I20260807 22:49:56.931936 11808 log.cc:1196] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/ffb215de628d4dec936b6ed74e354f68
Aug 07 22:49:56 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:49:56.932512 11808 ts_tablet_manager.cc:1939] T ffb215de628d4dec936b6ed74e354f68 P 06e5a2f0fc084270b07f1918ab3821ad: Deleting consensus metadata
I20260807 22:49:56.932576 11184 catalog_manager.cc:4841] TS b85a7a74f1a74243959fabe22f949dd1 (127.9.194.194:46513): tablet ffb215de628d4dec936b6ed74e354f68 (table test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]) successfully deleted
I20260807 22:49:56.933591 11181 catalog_manager.cc:4841] TS 06e5a2f0fc084270b07f1918ab3821ad (127.9.194.195:39753): tablet ffb215de628d4dec936b6ed74e354f68 (table test-table [id=a92a0e75f4e742d8af5d0c10623f46c0]) successfully deleted
I20260807 22:49:56.941220 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.909777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.9.194.193:43239)
0807 22:49:56.909996 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.910002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.915730 (+ 5728us) server_negotiation.cc:407] Connection header received
0807 22:49:56.916066 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.916075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.916165 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.916244 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:56.921227 (+ 4983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.922359 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.925327 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.925968 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.926064 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.934103 (+ 8039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:56.934116 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:56.934130 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:56.934175 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:56.937932 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.938740 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.938748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.938750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.938894 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.939304 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.939308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.939311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.939751 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:56.940066 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.940396 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.940662 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 22:49:56.952486 11415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33716
I20260807 22:49:56.982887 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.962787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.9.194.254:39961)
0807 22:49:56.970532 (+ 7745us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.970536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.970553 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:49:56.970607 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.970611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.970668 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.970800 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:56.972255 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.973059 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.974258 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.974706 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.974843 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:56.977290 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:56.977296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:56.977301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:56.977345 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:56.980138 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.980811 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.980817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.980827 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.980956 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:56.981330 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:56.981338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:56.981340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:56.981596 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:56.981867 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:56.982184 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:49:56.982415 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7631,"thread_start_us":7471,"threads_started":1}
W20260807 22:49:56.983593 11197 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49724
I20260807 22:49:56.986605 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11272
I20260807 22:49:57.008599 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:56.988473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.9.194.193:43239)
0807 22:49:56.988714 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:56.988723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:56.995951 (+ 7228us) server_negotiation.cc:407] Connection header received
0807 22:49:56.996178 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:56.996182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:56.996248 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:56.996365 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:56.997718 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:56.998512 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:56.999833 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:57.000407 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:57.000544 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:57.002782 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:57.002788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:57.002799 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:57.002839 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:57.005740 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.006411 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.006416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.006420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.006576 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.006956 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.006961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.006966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.007253 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:57.007589 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:57.007872 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 22:49:57.008124 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 22:49:57.009210 11415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33726
W20260807 22:49:57.009788 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11272: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:57.009985 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11272
I20260807 22:49:57.049535 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11424
Aug 07 22:49:57 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:49:57.067258 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:57.050399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.9.194.194:46513)
0807 22:49:57.050610 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:57.050616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:57.050638 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:49:57.050790 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:57.050796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:57.050896 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:57.051013 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:57.052228 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:57.053394 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:57.054620 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:57.055313 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:57.055516 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:57.059905 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:57.059923 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:57.059944 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:57.059994 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:57.064339 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.065012 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.065023 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.065027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.065186 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.065541 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.065551 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.065557 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.065994 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:57.066312 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:57.066626 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 22:49:57.066884 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 22:49:57.068207 11562 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51572
W20260807 22:49:57.068920 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11424: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:57.069119 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11424
I20260807 22:49:57.090296 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11573
Aug 07 22:49:57 dist-test-slave-5k54 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:49:57.106146 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:57.091067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.9.194.195:39753)
0807 22:49:57.091656 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:57.091661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:57.091677 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:49:57.091732 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:57.091735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:57.091794 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:57.091910 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:57.093268 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:57.094061 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:57.095270 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:57.095857 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:57.095954 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:57.100351 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:57.100367 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:57.100384 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:57.100425 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:57.103158 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.103961 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.103969 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.103971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.104116 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:57.104516 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:57.104522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:57.104525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:57.104918 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:57.105229 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:57.105540 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:49:57.105787 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":366,"threads_started":1}
W20260807 22:49:57.106787 11712 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476
W20260807 22:49:57.107415 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11573: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:57.107651 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11573
I20260807 22:49:57.123785 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11165
I20260807 22:49:57.125072 11240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:57.241545 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11165
2026-08-07T22:49:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4987 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:57 dist-test-slave-5k54 krb5kdc[11821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:57 dist-test-slave-5k54 krb5kdc[11821](info): set up 2 sockets
Aug 07 22:49:57 dist-test-slave-5k54 krb5kdc[11821](info): commencing operation
krb5kdc: starting...
W20260807 22:49:58.436976 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.008s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:49:58 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:49:58.637352 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41419
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:41419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:58.959625 11834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:58.959908 11834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:58.960080 11834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:58.967316 11834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:58.967473 11834 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:49:58.967574 11834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:58.967695 11834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:58.967765 11834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:58.967846 11834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:58.977820 11834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:41419
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:58.979141 11834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:58.980517 11834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:58.992277 11840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:58.992295 11841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:58.992363 11843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:58.993309 11834 server_base.cc:1034] running on GCE node
I20260807 22:49:58.994071 11834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:58.995350 11834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:58.996644 11834 hybrid_clock.cc:648] HybridClock initialized: now 1786142998996567 us; error 48 us; skew 500 ppm
I20260807 22:49:58.997005 11834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:58 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142998, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:59.000218 11834 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:49:59.001576 11834 webserver.cc:460] Webserver started at http://127.9.194.254:44493/ using document root <none> and password file <none>
I20260807 22:49:59.002223 11834 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:59.002310 11834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:59.002584 11834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:59.005595 11834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "a07f18a29dab4fbfaf1f608107cea281"
format_stamp: "Formatted at 2026-08-07 22:49:58 on dist-test-slave-5k54"
server_key: "e9fc9ae4af8e5449a33ccd31c49736c6"
server_key_iv: "961716f2fbede62e043217faec9c759d"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.006320 11834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "a07f18a29dab4fbfaf1f608107cea281"
format_stamp: "Formatted at 2026-08-07 22:49:58 on dist-test-slave-5k54"
server_key: "e9fc9ae4af8e5449a33ccd31c49736c6"
server_key_iv: "961716f2fbede62e043217faec9c759d"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.011509 11834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 22:49:59.014792 11851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:59.016203 11834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:59.016345 11834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "a07f18a29dab4fbfaf1f608107cea281"
format_stamp: "Formatted at 2026-08-07 22:49:58 on dist-test-slave-5k54"
server_key: "e9fc9ae4af8e5449a33ccd31c49736c6"
server_key_iv: "961716f2fbede62e043217faec9c759d"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.016522 11834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:59.030352 11834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:59.034224 11834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:59.034508 11834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:59.056502 11834 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:41419
I20260807 22:49:59.056514 11916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:41419 every 8 connection(s)
I20260807 22:49:59.057988 11834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:59.061865 11917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:59.065229 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11834
I20260807 22:49:59.065448 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:49:59.066120 9995 external_mini_cluster.cc:1428] Setting key c3d6b0ce85a47e638916e71beebd1cec
I20260807 22:49:59.074748 11917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Bootstrap starting.
I20260807 22:49:59.078040 11917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:59.079049 11917 log.cc:824] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:49:59 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:59.082391 11917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: No bootstrap required, opened a new log
I20260807 22:49:59.089419 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:59.068094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.9.194.254:41419)
0807 22:49:59.068864 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:59.068876 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:59.068918 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:49:59.070543 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:59.070564 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:59.070959 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:59.071290 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:59.073113 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.074331 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.075730 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.076440 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.076652 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:59.080783 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:59.080799 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:59.080816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:59.080858 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:59.084911 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.085622 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.085632 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.085640 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.085863 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.086193 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.086198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.086202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.086937 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:59.087251 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:59.087557 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:49:59.088021 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":362,"threads_started":1}
I20260807 22:49:59.092360 11917 raft_consensus.cc:348] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } }
I20260807 22:49:59.092681 11917 raft_consensus.cc:374] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:59.092760 11917 raft_consensus.cc:729] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07f18a29dab4fbfaf1f608107cea281, State: Initialized, Role: FOLLOWER
I20260807 22:49:59.093295 11917 consensus_queue.cc:260] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } }
I20260807 22:49:59.093492 11917 raft_consensus.cc:388] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:59.093588 11917 raft_consensus.cc:482] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:59.093703 11917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:59.095866 11917 raft_consensus.cc:504] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } }
I20260807 22:49:59.096412 11917 leader_election.cc:302] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a07f18a29dab4fbfaf1f608107cea281; no voters:
I20260807 22:49:59.097021 11917 leader_election.cc:288] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:59.097105 11922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:59.097841 11922 raft_consensus.cc:686] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [term 1 LEADER]: Becoming Leader. State: Replica: a07f18a29dab4fbfaf1f608107cea281, State: Running, Role: LEADER
I20260807 22:49:59.098398 11922 consensus_queue.cc:237] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } }
I20260807 22:49:59.099109 11917 sys_catalog.cc:564] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:59.101269 11924 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a07f18a29dab4fbfaf1f608107cea281. Latest consensus state: current_term: 1 leader_uuid: "a07f18a29dab4fbfaf1f608107cea281" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } } }
I20260807 22:49:59.101713 11924 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:59.102241 11923 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a07f18a29dab4fbfaf1f608107cea281" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07f18a29dab4fbfaf1f608107cea281" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41419 } } }
I20260807 22:49:59.102643 11923 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:59.104535 11927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:59.109557 11927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:59.117512 11927 catalog_manager.cc:1347] Generated new cluster ID: 983ff6780e0942ccabce8bc0407c229d
I20260807 22:49:59.117662 11927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:59.156018 11927 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:59.157608 11927 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:59.167263 11927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Generated new TSK 0
I20260807 22:49:59.168130 11927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:49:59.225065 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41419
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:59.578083 11944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:59.578379 11944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:59.578481 11944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:59.585955 11944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:59.586071 11944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:59.586159 11944 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:49:59.586211 11944 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:49:59.586288 11944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:49:59.596472 11944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:59.597685 11944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:59.598960 11944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:59.612409 11954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:59.612541 11949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:59.612445 11951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:59.613461 11944 server_base.cc:1034] running on GCE node
I20260807 22:49:59.614094 11944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:59.614995 11944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:59.616256 11944 hybrid_clock.cc:648] HybridClock initialized: now 1786142999616214 us; error 52 us; skew 500 ppm
I20260807 22:49:59.616604 11944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:59 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142999, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:59.625353 11944 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:49:59.626828 11944 webserver.cc:460] Webserver started at http://127.9.194.193:33489/ using document root <none> and password file <none>
I20260807 22:49:59.627543 11944 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:59.627651 11944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:59.627919 11944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:59.630719 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "97970aff3cc649f1a31fc735739159ce"
format_stamp: "Formatted at 2026-08-07 22:49:59 on dist-test-slave-5k54"
server_key: "26223257ad50a53bdc6c110468d9b1b2"
server_key_iv: "e4e8b7538364aaa02f272d6414d04e2e"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.631486 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "97970aff3cc649f1a31fc735739159ce"
format_stamp: "Formatted at 2026-08-07 22:49:59 on dist-test-slave-5k54"
server_key: "26223257ad50a53bdc6c110468d9b1b2"
server_key_iv: "e4e8b7538364aaa02f272d6414d04e2e"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.636570 11944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:49:59.639858 11961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:59.641179 11944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:49:59.641346 11944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "97970aff3cc649f1a31fc735739159ce"
format_stamp: "Formatted at 2026-08-07 22:49:59 on dist-test-slave-5k54"
server_key: "26223257ad50a53bdc6c110468d9b1b2"
server_key_iv: "e4e8b7538364aaa02f272d6414d04e2e"
server_key_version: "encryptionkey@0"
I20260807 22:49:59.641541 11944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:59.666611 11944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:59.671155 11944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:59.671612 11944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:59.674388 11944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:49:59.674520 11944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:59.674635 11944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:49:59.674726 11944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:49:59 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142999, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:49:59.698890 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:59.675149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:54861 (local address 127.9.194.254:41419)
0807 22:49:59.675694 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:59.675699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:59.676972 (+ 1273us) server_negotiation.cc:407] Connection header received
0807 22:49:59.678968 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:59.678974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:59.679069 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:59.679203 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:59.681550 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.682886 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.685583 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.686435 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.686591 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:59.691040 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:59.691049 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:59.691057 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:59.691109 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:59.695035 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.696005 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.696010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.696014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.696188 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.696788 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.696795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.696799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.697232 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:59.697415 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:59.698244 (+ 829us) server_negotiation.cc:299] Negotiation successful
0807 22:49:59.698483 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":336,"threads_started":1}
I20260807 22:49:59.700567 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:59.675951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.193:54861)
0807 22:49:59.676808 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:59.676850 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:59.678691 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:59.679510 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:59.679527 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:59.680356 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:59.681246 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:59.681276 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.683094 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.683102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:59.685418 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:59.685435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.686730 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.686742 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:59.686917 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:59.687752 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:59.687774 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:59.690715 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:59.695193 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:59.695203 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:59.695220 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:59.695816 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:59.696329 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:59.696338 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:59.696344 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:59.696644 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:59.697397 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:59.697405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:59.697832 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 22:49:59.698114 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 22:49:59.698691 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":363,"threads_started":1}
I20260807 22:49:59.728080 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:59.710559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:33635 (local address 127.9.194.254:41419)
0807 22:49:59.710776 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:59.710785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:59.711812 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 22:49:59.712056 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:59.712063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:59.712137 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:59.712249 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:59.713802 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.715119 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.717492 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.718061 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.718214 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:59.720696 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:59.720701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:59.720706 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:59.720762 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:59.724481 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.725681 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.725689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.725695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.725873 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:59.726319 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:59.726324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:59.726328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:59.726770 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:59.726949 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:59.727375 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 22:49:59.727648 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:49:59.728242 12038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:59.710627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.193:33635)
0807 22:49:59.711660 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:59.711691 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:59.711894 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:59.712284 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:59.712288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:59.713102 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:59.713612 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:59.713632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.715274 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.715282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:59.717317 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:59.717340 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:59.718263 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:59.718269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:59.718508 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:59.719335 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:59.719354 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:59.720454 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:59.724668 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:59.724673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:59.724680 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:59.725456 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:59.726013 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:59.726022 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:59.726026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:59.726176 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:59.726904 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:59.726915 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:59.727110 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 22:49:59.727259 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 22:49:59.727660 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":537,"threads_started":1}
I20260807 22:49:59.737118 11944 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:38405
I20260807 22:49:59.737169 12087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:38405 every 8 connection(s)
I20260807 22:49:59.738468 11944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:49:59.740711 12088 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41419
I20260807 22:49:59.740919 12088 heartbeater.cc:464] Registering TS with master...
I20260807 22:49:59.741516 12088 heartbeater.cc:511] Master 127.9.194.254:41419 requested a full tablet report, sending...
I20260807 22:49:59.743583 11868 ts_manager.cc:194] Registered new tserver with Master: 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405)
I20260807 22:49:59.745113 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:54861
I20260807 22:49:59.748697 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 11944
I20260807 22:49:59.748914 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:49:59.749588 9995 external_mini_cluster.cc:1428] Setting key 0c08187d877a8f11f6463b2e42f39b98
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:49:59.796164 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41419
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:00.123411 12092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:00.123782 12092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:00.123972 12092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:00.131197 12092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:00.131345 12092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:00.131537 12092 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:50:00.131629 12092 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:50:00.131740 12092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:00.141371 12092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:00.142699 12092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:00.144061 12092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:00.157034 12098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:00.157007 12100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:00.157342 12097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:00.157792 12092 server_base.cc:1034] running on GCE node
I20260807 22:50:00.158504 12092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:00.159597 12092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:00.160818 12092 hybrid_clock.cc:648] HybridClock initialized: now 1786143000160770 us; error 45 us; skew 500 ppm
I20260807 22:50:00.161163 12092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:00.164462 12092 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:00.165979 12092 webserver.cc:460] Webserver started at http://127.9.194.194:33925/ using document root <none> and password file <none>
I20260807 22:50:00.166683 12092 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:00.166828 12092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:00.167138 12092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:00.170070 12092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "42c1677b2ab9471e9bfe03fef0cca643"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "39ddab025b4ee8117c7d357cc55f602c"
server_key_iv: "c1641a394d927f8e2b7ae58505a3aa48"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.170831 12092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "42c1677b2ab9471e9bfe03fef0cca643"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "39ddab025b4ee8117c7d357cc55f602c"
server_key_iv: "c1641a394d927f8e2b7ae58505a3aa48"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.175966 12092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:50:00.179237 12108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:00.180619 12092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:00.180823 12092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "42c1677b2ab9471e9bfe03fef0cca643"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "39ddab025b4ee8117c7d357cc55f602c"
server_key_iv: "c1641a394d927f8e2b7ae58505a3aa48"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.181038 12092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:00.203743 12092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:00.208379 12092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:00.208666 12092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:00.211437 12092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:00.211560 12092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 22:50:00.211675 12092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:00.211776 12092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:00.237380 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.212020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:47499 (local address 127.9.194.254:41419)
0807 22:50:00.212227 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.212233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.213840 (+ 1607us) server_negotiation.cc:407] Connection header received
0807 22:50:00.216283 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.216290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.216382 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.216500 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:00.219075 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.220226 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.223906 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.224483 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.224645 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.230012 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:00.230019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:00.230027 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:00.230077 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:00.233477 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.234521 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.234525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.234527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.234642 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.235084 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.235089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.235094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.235368 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:00.235559 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.236565 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.236800 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:50:00.238969 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.212695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.194:47499)
0807 22:50:00.213644 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.213684 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.215924 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.216806 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.216822 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.217784 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:00.218775 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.218816 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.220415 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.220420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.223648 (+ 3228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.223668 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.224817 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.224829 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.225208 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.226044 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:00.226092 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:00.229728 (+ 3636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:00.233632 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.233644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.233669 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.234363 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.234758 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.234762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.234772 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.234961 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.235521 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:00.235529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:00.236098 (+ 569us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.236466 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.236994 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":458,"threads_started":1}
I20260807 22:50:00.262795 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.245680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:56201 (local address 127.9.194.254:41419)
0807 22:50:00.245927 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.245935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.246564 (+ 629us) server_negotiation.cc:407] Connection header received
0807 22:50:00.246662 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.246670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.246769 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.246935 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:00.248087 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.249367 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.252138 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.252867 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.253020 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.255625 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:00.255633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:00.255638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:00.255679 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:00.259229 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.260328 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.260335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.260341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.260515 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.261012 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.261021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.261027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.261604 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:00.261794 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.262218 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.262440 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:50:00.262904 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.245728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.194:56201)
0807 22:50:00.246410 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.246430 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.246552 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.246918 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.246920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.247399 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:00.247911 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.247929 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.249557 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.249564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.251916 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.251936 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.253312 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.253323 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.253583 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.254449 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:00.254482 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:00.255369 (+ 887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:00.259427 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.259476 (+ 49us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.259481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.260156 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.260638 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.260647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.260650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.260849 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.261748 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:00.261752 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:00.261921 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.262109 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.262443 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":368,"threads_started":1}
I20260807 22:50:00.266366 12092 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:44371
I20260807 22:50:00.266702 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:44371 every 8 connection(s)
I20260807 22:50:00.267884 12092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:00.269825 12235 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41419
I20260807 22:50:00.270061 12235 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:00.270675 12235 heartbeater.cc:511] Master 127.9.194.254:41419 requested a full tablet report, sending...
I20260807 22:50:00.272032 11868 ts_manager.cc:194] Registered new tserver with Master: 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371)
I20260807 22:50:00.272831 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:47499
I20260807 22:50:00.276854 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12092
I20260807 22:50:00.277061 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:00.277583 9995 external_mini_cluster.cc:1428] Setting key 13f781287164c23b56571f56ef754a06
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:00.323159 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41419
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:00.650105 12239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:00.650430 12239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:00.650617 12239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:00.657691 12239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:00.657846 12239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:00.658028 12239 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:50:00.658120 12239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:50:00.658238 12239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:00.668355 12239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:00.669719 12239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:00.671121 12239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:00.685771 12245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:00.685806 12247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:00.685845 12244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:00.691872 12239 server_base.cc:1034] running on GCE node
I20260807 22:50:00.692464 12239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:00.693362 12239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:00.694572 12239 hybrid_clock.cc:648] HybridClock initialized: now 1786143000694509 us; error 52 us; skew 500 ppm
I20260807 22:50:00.694882 12239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:00.698024 12239 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:00.699414 12239 webserver.cc:460] Webserver started at http://127.9.194.195:46669/ using document root <none> and password file <none>
I20260807 22:50:00.700194 12239 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:00.700299 12239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:00.700564 12239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:00.703649 12239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "4b286375abc2ec3934db9f7ddd9b6042"
server_key_iv: "871151d271f1ae894e75d25a7b5d5db5"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.704384 12239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "4b286375abc2ec3934db9f7ddd9b6042"
server_key_iv: "871151d271f1ae894e75d25a7b5d5db5"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.709463 12239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:00.712704 12255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:00.713968 12239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:00.714128 12239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
format_stamp: "Formatted at 2026-08-07 22:50:00 on dist-test-slave-5k54"
server_key: "4b286375abc2ec3934db9f7ddd9b6042"
server_key_iv: "871151d271f1ae894e75d25a7b5d5db5"
server_key_version: "encryptionkey@0"
I20260807 22:50:00.714298 12239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:00.738507 12239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:00.742835 12239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:00.743131 12239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:00.746062 12239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:00.746186 12239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:00.746332 12239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:00.746461 12239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 22:50:00.747886 12088 heartbeater.cc:503] Master 127.9.194.254:41419 was elected leader, sending a full tablet report...
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:00.772245 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.746588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:57471 (local address 127.9.194.254:41419)
0807 22:50:00.746782 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.746790 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.748357 (+ 1567us) server_negotiation.cc:407] Connection header received
0807 22:50:00.750995 (+ 2638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.751001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.751094 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.751222 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:00.753489 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.754542 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.757156 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.757755 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.757923 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.763549 (+ 5626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:00.763554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:00.763558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:00.763594 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:00.767766 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.768854 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.768863 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.768867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.769078 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.769782 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.769790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.769806 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.770165 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:00.770331 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.771643 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.771884 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:50:00.773540 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.747218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.195:57471)
0807 22:50:00.748196 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.748248 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.750659 (+ 2411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.751411 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.751465 (+ 54us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.752443 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:00.753288 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.753326 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.754715 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.754722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.756996 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.757012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.758059 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.758073 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.758488 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.759389 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:00.759452 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:00.763181 (+ 3729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:00.767930 (+ 4749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.767940 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.767957 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.768707 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.769221 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.769229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.769234 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.769587 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.770330 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:00.770339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:00.770852 (+ 513us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.771255 (+ 403us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.771884 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":433,"threads_started":1}
I20260807 22:50:00.791371 12239 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:38363
I20260807 22:50:00.791450 12381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:38363 every 8 connection(s)
I20260807 22:50:00.792619 12239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:00.794775 12382 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41419
I20260807 22:50:00.795810 12382 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:00.795791 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12239
I20260807 22:50:00.795992 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:00.796677 9995 external_mini_cluster.cc:1428] Setting key 6102495f81e8c6131ef1b557f7b14a68
I20260807 22:50:00.796777 12382 heartbeater.cc:511] Master 127.9.194.254:41419 requested a full tablet report, sending...
I20260807 22:50:00.798007 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.781360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:38569 (local address 127.9.194.254:41419)
0807 22:50:00.781508 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.781512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.782348 (+ 836us) server_negotiation.cc:407] Connection header received
0807 22:50:00.782553 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.782559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.782625 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.782732 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:00.784490 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.785733 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.788252 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.789019 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.789176 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.791704 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:00.791709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:00.791712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:00.791742 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:00.795125 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.795922 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.795928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.795933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.796120 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.796527 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.796533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.796536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.796865 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:00.797007 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.797404 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.797621 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:50:00.798602 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.781484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.9.194.195:38569)
0807 22:50:00.782220 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.782237 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.782421 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.782765 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.782772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.783589 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:00.784258 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.784281 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.785955 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.785963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.788059 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.788081 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.789276 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.789293 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.789486 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.790432 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:00.790468 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:00.791407 (+ 939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:00.795236 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.795240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.795242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.795796 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.796247 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:00.796252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:00.796256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:00.796403 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:00.796973 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:00.796977 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:00.797150 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.797284 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.797611 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":437,"threads_started":1}
I20260807 22:50:00.798605 11868 ts_manager.cc:194] Registered new tserver with Master: 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:00.799976 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:57471
I20260807 22:50:00.800230 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:50:00 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:00.851923 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.837882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.9.194.254:41419)
0807 22:50:00.838130 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.838136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.838322 (+ 186us) server_negotiation.cc:407] Connection header received
0807 22:50:00.838549 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.838554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.838617 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.838723 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:00.840201 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.840953 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.842232 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.842676 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.842816 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.846787 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:00.846792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:00.846795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:00.846831 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:00.849504 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.850175 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.850179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.850181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.850302 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:00.850640 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:00.850643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:00.850645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:00.850918 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:00.851121 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.851364 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.851630 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:50:00.856135 11866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:00.863077 11866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:00.885890 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.876193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38405 (local address 127.0.0.1:41692)
0807 22:50:00.876717 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.876750 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.876954 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.877848 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.877853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.877919 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:00.878238 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.878247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.880941 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.880947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.882936 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.882958 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.884797 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.884821 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.885037 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.885104 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.885219 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.885457 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":1419,"threads_started":1}
I20260807 22:50:00.887002 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.876690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.9.194.193:38405)
0807 22:50:00.877209 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.877218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.877248 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:50:00.877346 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.877353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.877631 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.877836 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:00.879515 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.880803 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.883096 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.884543 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.886200 (+ 1657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.886444 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.886527 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.886698 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":2380,"threads_started":1}
I20260807 22:50:00.890318 12014 tablet_service.cc:1467] Processing CreateTablet for tablet 8479143d22fe4232a63ba56c78f951fd (DEFAULT_TABLE table=test-table [id=181de52e8dc847439771c25705c22763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:00.891880 12014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8479143d22fe4232a63ba56c78f951fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:00.892853 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.878420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:38363 (local address 127.0.0.1:34172)
0807 22:50:00.883840 (+ 5420us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.883872 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.883978 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.885246 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.885250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.885290 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:00.885715 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.885728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.887907 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.887912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.890116 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.890132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.891945 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.891952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.892207 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.892253 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.892327 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.892484 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5283,"mutex_wait_us":47,"thread_start_us":5169,"threads_started":1}
I20260807 22:50:00.892969 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.878788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.9.194.195:38363)
0807 22:50:00.884592 (+ 5804us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.884602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.884636 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:50:00.884735 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.884744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.885027 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.885938 (+ 911us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:00.886286 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.887772 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.890301 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.891717 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.892195 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.892434 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.892508 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.892653 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5621,"thread_start_us":5412,"threads_started":1}
I20260807 22:50:00.896265 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.884247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37820 (local address 127.9.194.194:44371)
0807 22:50:00.884850 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.884858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.885791 (+ 933us) server_negotiation.cc:407] Connection header received
0807 22:50:00.886309 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.886318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.886576 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.886774 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:00.888765 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.890218 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.892349 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.893770 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.893924 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.894120 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.895765 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.895932 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":307,"threads_started":1}
I20260807 22:50:00.896256 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.884884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.0.0.1:37820)
0807 22:50:00.885643 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.885662 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.885813 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.888097 (+ 2284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.888104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.888136 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:00.888569 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.888581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.890390 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.890395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.892177 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.892192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.893993 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.894002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.895588 (+ 1586us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.895611 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.895714 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.895839 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":18,"thread_start_us":435,"threads_started":1}
I20260807 22:50:00.896867 12309 tablet_service.cc:1467] Processing CreateTablet for tablet 8479143d22fe4232a63ba56c78f951fd (DEFAULT_TABLE table=test-table [id=181de52e8dc847439771c25705c22763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:00.898340 12309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8479143d22fe4232a63ba56c78f951fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:00.899278 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 8479143d22fe4232a63ba56c78f951fd (DEFAULT_TABLE table=test-table [id=181de52e8dc847439771c25705c22763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:00.900473 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8479143d22fe4232a63ba56c78f951fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:00.907135 12401 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Bootstrap starting.
I20260807 22:50:00.910844 12401 tablet_bootstrap.cc:654] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:00.911720 12402 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Bootstrap starting.
I20260807 22:50:00.912139 12401 log.cc:824] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:00.914505 12404 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Bootstrap starting.
I20260807 22:50:00.915674 12402 tablet_bootstrap.cc:654] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:00.916976 12402 log.cc:824] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:00.917929 12401 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: No bootstrap required, opened a new log
I20260807 22:50:00.917939 12404 tablet_bootstrap.cc:654] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:00.918222 12401 ts_tablet_manager.cc:1397] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 22:50:00.919202 12404 log.cc:824] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:00.920120 12402 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: No bootstrap required, opened a new log
I20260807 22:50:00.920408 12402 ts_tablet_manager.cc:1397] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:50:00.922144 12404 tablet_bootstrap.cc:492] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: No bootstrap required, opened a new log
I20260807 22:50:00.922400 12404 ts_tablet_manager.cc:1397] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 22:50:00.927924 12402 raft_consensus.cc:348] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.928279 12402 raft_consensus.cc:374] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:00.928418 12402 raft_consensus.cc:729] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Initialized, Role: FOLLOWER
I20260807 22:50:00.928812 12401 raft_consensus.cc:348] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.929031 12402 consensus_queue.cc:260] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.929219 12401 raft_consensus.cc:374] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:00.929322 12401 raft_consensus.cc:729] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Initialized, Role: FOLLOWER
I20260807 22:50:00.930141 12382 heartbeater.cc:503] Master 127.9.194.254:41419 was elected leader, sending a full tablet report...
I20260807 22:50:00.930013 12401 consensus_queue.cc:260] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.932863 12401 ts_tablet_manager.cc:1428] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 22:50:00.933167 12402 ts_tablet_manager.cc:1428] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 22:50:00.932888 12404 raft_consensus.cc:348] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.933300 12404 raft_consensus.cc:374] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:00.933372 12404 raft_consensus.cc:729] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Initialized, Role: FOLLOWER
I20260807 22:50:00.933984 12404 consensus_queue.cc:260] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.935070 12235 heartbeater.cc:503] Master 127.9.194.254:41419 was elected leader, sending a full tablet report...
I20260807 22:50:00.935938 12404 ts_tablet_manager.cc:1428] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260807 22:50:00.955235 12409 raft_consensus.cc:482] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:00.955554 12409 raft_consensus.cc:504] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.957067 12409 leader_election.cc:288] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:00.963635 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.957543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:57899 (local address 127.9.194.193:38405)
0807 22:50:00.957807 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.957816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.958054 (+ 238us) server_negotiation.cc:407] Connection header received
0807 22:50:00.958357 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.958364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.958459 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.958512 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:00.959342 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.960090 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.961919 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.962685 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.962861 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.962942 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.963233 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.963365 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:50:00.963778 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.957291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38405 (local address 127.9.194.194:57899)
0807 22:50:00.957935 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.957950 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.958170 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.958698 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.958702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.958733 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:00.959192 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.959203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.960215 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.960223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.961757 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.961768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.962916 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.962921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.963076 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.963099 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.963205 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.963330 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":22,"thread_start_us":286,"threads_started":1}
I20260807 22:50:00.963809 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.958151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:33091 (local address 127.9.194.195:38363)
0807 22:50:00.958373 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:00.958379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:00.958708 (+ 329us) server_negotiation.cc:407] Connection header received
0807 22:50:00.958810 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:00.958815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:00.958895 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:00.958991 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:00.959641 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.960315 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.962088 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.962859 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.963064 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.963134 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:00.963400 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:50:00.963544 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:50:00.963943 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:00.958029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:38363 (local address 127.9.194.194:33091)
0807 22:50:00.958598 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:00.958612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:00.958713 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:00.959007 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:00.959010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:00.959030 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:00.959414 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.959477 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.960476 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.960481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.961934 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:00.961946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:00.963091 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:00.963097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:00.963243 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:00.963267 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:50:00.963371 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 22:50:00.963525 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":20,"thread_start_us":305,"threads_started":1}
I20260807 22:50:00.964965 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8479143d22fe4232a63ba56c78f951fd" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce" is_pre_election: true
I20260807 22:50:00.964963 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8479143d22fe4232a63ba56c78f951fd" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" is_pre_election: true
I20260807 22:50:00.965523 12040 raft_consensus.cc:2454] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 0.
I20260807 22:50:00.965528 12331 raft_consensus.cc:2454] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 0.
I20260807 22:50:00.966205 12109 leader_election.cc:302] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:00.966564 12409 raft_consensus.cc:2781] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:00.966723 12409 raft_consensus.cc:482] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:00.966830 12409 raft_consensus.cc:3037] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:00.969447 12409 raft_consensus.cc:504] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.970067 12409 leader_election.cc:288] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 election: Requested vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:00.970501 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8479143d22fe4232a63ba56c78f951fd" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce"
I20260807 22:50:00.970659 12040 raft_consensus.cc:3037] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:00.970616 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8479143d22fe4232a63ba56c78f951fd" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
I20260807 22:50:00.970845 12331 raft_consensus.cc:3037] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:00.973362 12040 raft_consensus.cc:2454] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 1.
I20260807 22:50:00.973623 12331 raft_consensus.cc:2454] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 1.
I20260807 22:50:00.973754 12109 leader_election.cc:302] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:00.974032 12409 raft_consensus.cc:2781] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:00.974483 12409 raft_consensus.cc:686] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 LEADER]: Becoming Leader. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Running, Role: LEADER
I20260807 22:50:00.974993 12409 consensus_queue.cc:237] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } }
I20260807 22:50:00.979525 11868 catalog_manager.cc:5495] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } health_report { overall_health: HEALTHY } } }
W20260807 22:50:00.990833 12089 tablet.cc:2369] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:50:01.019870 12236 tablet.cc:2369] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:50:01 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142998, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:01.035470 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.022278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.9.194.254:41419)
0807 22:50:01.022473 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:01.022478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:01.023018 (+ 540us) server_negotiation.cc:407] Connection header received
0807 22:50:01.023281 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:01.023285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:01.023347 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:01.023478 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:01.024652 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.025402 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.026673 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.027094 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.027183 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.030453 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:01.030458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:01.030461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:01.030488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:01.032880 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:01.033549 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:01.033552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:01.033554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:01.033702 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:01.034051 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:01.034054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:01.034056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:01.034370 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:01.034482 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:01.034855 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 22:50:01.035161 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":69}
I20260807 22:50:01.035499 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.022310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41419 (local address 127.0.0.1:54792)
0807 22:50:01.022917 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:01.022932 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:01.023056 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:01.023537 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:01.023541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:01.024250 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:01.024529 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.024538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.025528 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.025532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.026559 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.026570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.027320 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.027326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.027467 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.028090 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:01.028108 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:01.030292 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:01.033033 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:01.033043 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:01.033048 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:01.033451 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:01.033812 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:01.033815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:01.033817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:01.033962 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:01.034482 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:01.034488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:01.034631 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 22:50:01.034761 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 22:50:01.035034 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":308,"threads_started":1}
I20260807 22:50:01.040480 11868 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:54792:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:50:01.041548 11868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 22:50:01.044799 12383 tablet.cc:2369] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:01.048636 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8b6e3ca4dc4c7a83a4539df864daef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49c44baaa8ca4baeabd4f6e005a140f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:01.048481 12014 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8b6e3ca4dc4c7a83a4539df864daef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49c44baaa8ca4baeabd4f6e005a140f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:01.048803 12309 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8b6e3ca4dc4c7a83a4539df864daef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49c44baaa8ca4baeabd4f6e005a140f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:01.049474 12014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8b6e3ca4dc4c7a83a4539df864daef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.049469 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8b6e3ca4dc4c7a83a4539df864daef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.049583 12309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8b6e3ca4dc4c7a83a4539df864daef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.054880 12402 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: Bootstrap starting.
I20260807 22:50:01.056536 12402 tablet_bootstrap.cc:654] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.056525 12404 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: Bootstrap starting.
I20260807 22:50:01.056541 12401 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: Bootstrap starting.
I20260807 22:50:01.058125 12401 tablet_bootstrap.cc:654] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.058125 12404 tablet_bootstrap.cc:654] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.070252 12402 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: No bootstrap required, opened a new log
I20260807 22:50:01.070461 12402 ts_tablet_manager.cc:1397] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260807 22:50:01.071256 12402 raft_consensus.cc:348] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.071659 12402 raft_consensus.cc:374] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.071776 12402 raft_consensus.cc:729] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.072041 12402 consensus_queue.cc:260] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.072697 12402 ts_tablet_manager.cc:1428] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:50:01.072536 12407 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.072933 12407 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.073876 12401 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: No bootstrap required, opened a new log
I20260807 22:50:01.074052 12401 ts_tablet_manager.cc:1397] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.011s
I20260807 22:50:01.074734 12401 raft_consensus.cc:348] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.074991 12401 raft_consensus.cc:374] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.075084 12401 raft_consensus.cc:729] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.076985 12404 tablet_bootstrap.cc:492] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: No bootstrap required, opened a new log
I20260807 22:50:01.077147 12404 ts_tablet_manager.cc:1397] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.003s
I20260807 22:50:01.077790 12404 raft_consensus.cc:348] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.077967 12404 raft_consensus.cc:374] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.078088 12404 raft_consensus.cc:729] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.078317 12404 consensus_queue.cc:260] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.078739 12409 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.078930 12404 ts_tablet_manager.cc:1428] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 22:50:01.079057 12409 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.075354 12401 consensus_queue.cc:260] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.080121 12401 ts_tablet_manager.cc:1428] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 22:50:01.080016 12408 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.080379 12408 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.383327 12409 consensus_queue.cc:1035] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:01.392649 12412 consensus_queue.cc:1035] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:01.428758 12412 raft_consensus.cc:482] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:01.428993 12412 raft_consensus.cc:504] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.429551 12412 leader_election.cc:288] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:01.430208 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a8b6e3ca4dc4c7a83a4539df864daef" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce" is_pre_election: true
I20260807 22:50:01.430392 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a8b6e3ca4dc4c7a83a4539df864daef" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" is_pre_election: true
I20260807 22:50:01.430532 12040 raft_consensus.cc:2454] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 0.
I20260807 22:50:01.430676 12331 raft_consensus.cc:2454] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 0.
I20260807 22:50:01.431079 12109 leader_election.cc:302] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:01.431471 12412 raft_consensus.cc:2781] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:01.431572 12412 raft_consensus.cc:482] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:01.431696 12412 raft_consensus.cc:3037] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:01.433727 12412 raft_consensus.cc:504] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.434262 12412 leader_election.cc:288] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 election: Requested vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:01.434852 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a8b6e3ca4dc4c7a83a4539df864daef" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce"
I20260807 22:50:01.434942 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a8b6e3ca4dc4c7a83a4539df864daef" candidate_uuid: "42c1677b2ab9471e9bfe03fef0cca643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
I20260807 22:50:01.435110 12040 raft_consensus.cc:3037] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:01.435181 12331 raft_consensus.cc:3037] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:01.437304 12331 raft_consensus.cc:2454] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 1.
I20260807 22:50:01.437463 12040 raft_consensus.cc:2454] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42c1677b2ab9471e9bfe03fef0cca643 in term 1.
I20260807 22:50:01.437736 12109 leader_election.cc:302] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229c45515b934dd4af45b0ea1a4bd4a9, 42c1677b2ab9471e9bfe03fef0cca643; no voters:
I20260807 22:50:01.438086 12412 raft_consensus.cc:2781] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:01.438431 12412 raft_consensus.cc:686] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 LEADER]: Becoming Leader. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Running, Role: LEADER
I20260807 22:50:01.438684 12412 consensus_queue.cc:237] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.440223 12409 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: TxnStatusTablet state changed. Reason: New leader 42c1677b2ab9471e9bfe03fef0cca643. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.440479 12409 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:01.441198 12429 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:50:01.441526 12429 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:50:01.441881 11866 catalog_manager.cc:5495] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:01.525781 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.518742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.0.0.1:37828)
0807 22:50:01.519003 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:01.519028 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:01.519228 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:01.520103 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:01.520108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:01.520122 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:01.520352 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.520362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.521820 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.521826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.522928 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.522941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.523844 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.523851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.524038 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.524144 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:01.525191 (+ 1047us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:01.525215 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:50:01.525306 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:50:01.525475 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:50:01.525789 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.518879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.9.194.194:44371)
0807 22:50:01.519602 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:01.519607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:01.519623 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:01.519767 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:01.519772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:01.519891 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:01.520080 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:01.520568 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.521664 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.523086 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.523664 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.523808 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.524417 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:01.525058 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:01.525236 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:01.525345 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:50:01.525435 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":349,"threads_started":1}
I20260807 22:50:01.533162 12040 raft_consensus.cc:1261] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Refusing update from remote peer 42c1677b2ab9471e9bfe03fef0cca643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:01.533190 12331 raft_consensus.cc:1261] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Refusing update from remote peer 42c1677b2ab9471e9bfe03fef0cca643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:01.533885 12412 consensus_queue.cc:1035] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:01.534257 12409 consensus_queue.cc:1035] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:01.541877 12408 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: TxnStatusTablet state changed. Reason: New leader 42c1677b2ab9471e9bfe03fef0cca643. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.542214 12408 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.544533 12424 mvcc.cc:205] Tried to move back new op lower bound from 7316041734271856640 to 7316041733894881280. Current Snapshot: MvccSnapshot[applied={T|T < 7316041734271856640}]
I20260807 22:50:01.545445 12425 mvcc.cc:205] Tried to move back new op lower bound from 7316041734271856640 to 7316041733894881280. Current Snapshot: MvccSnapshot[applied={T|T < 7316041734271856640}]
I20260807 22:50:01.545852 12407 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: TxnStatusTablet state changed. Reason: New leader 42c1677b2ab9471e9bfe03fef0cca643. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.546146 12407 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.546444 12408 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.546680 12408 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 97970aff3cc649f1a31fc735739159ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.551883 12412 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.552222 12412 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:01.557194 12407 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.557476 12407 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 229c45515b934dd4af45b0ea1a4bd4a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:01.561328 12409 tablet_replica.cc:439] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42c1677b2ab9471e9bfe03fef0cca643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } } }
I20260807 22:50:01.561687 12409 tablet_replica.cc:442] T 3a8b6e3ca4dc4c7a83a4539df864daef P 42c1677b2ab9471e9bfe03fef0cca643: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:01.572149 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.566082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.9.194.194:44371)
0807 22:50:01.566247 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:01.566251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:01.566810 (+ 559us) server_negotiation.cc:407] Connection header received
0807 22:50:01.567053 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:01.567063 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:01.567129 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:01.567233 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:01.567770 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.568634 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.570214 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.570715 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.570863 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.571316 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:01.571480 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:01.571625 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:01.571774 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 22:50:01.571861 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:50:01.590464 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.583905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:58649 (local address 127.9.194.194:44371)
0807 22:50:01.584069 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:01.584074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:01.584878 (+ 804us) server_negotiation.cc:407] Connection header received
0807 22:50:01.585146 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:01.585158 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:01.585247 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:01.585352 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:01.585945 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.586832 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.588493 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.588957 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.589103 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.589601 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:01.589705 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:01.589815 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:01.589956 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 22:50:01.590052 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":49}
I20260807 22:50:01.590575 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:01.583996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.9.194.194:58649)
0807 22:50:01.584762 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:01.584779 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:01.584959 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:01.585360 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:01.585364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:01.585376 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:01.585686 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.585722 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.587024 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.587030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.588336 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:01.588357 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:01.589116 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:01.589121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:01.589338 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:01.589397 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:01.589814 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:01.589828 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:50:01.589937 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 22:50:01.590089 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":40,"thread_start_us":409,"threads_started":1}
I20260807 22:50:01.664744 11866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:01.665084 11866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:01.668990 11866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Sending DeleteTablet for 3 replicas of tablet 8479143d22fe4232a63ba56c78f951fd
I20260807 22:50:01.670173 12014 tablet_service.cc:1514] Processing DeleteTablet for tablet 8479143d22fe4232a63ba56c78f951fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:01 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41692
I20260807 22:50:01.670403 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet 8479143d22fe4232a63ba56c78f951fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:01 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37820
I20260807 22:50:01.670446 12309 tablet_service.cc:1514] Processing DeleteTablet for tablet 8479143d22fe4232a63ba56c78f951fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:01 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:34172
I20260807 22:50:01.672801 11866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:01.674423 11866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:01.681761 12450 tablet_replica.cc:330] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: stopping tablet replica
I20260807 22:50:01.682397 12450 raft_consensus.cc:2229] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:01.683213 12450 raft_consensus.cc:2258] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:01.685846 12450 ts_tablet_manager.cc:1905] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:01.688997 12013 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbdad21d6e6478f88390a542ef95211 (DEFAULT_TABLE table=test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:01.689713 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbdad21d6e6478f88390a542ef95211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.691903 12448 tablet_replica.cc:330] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: stopping tablet replica
I20260807 22:50:01.692356 12448 raft_consensus.cc:2229] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:01.692296 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbdad21d6e6478f88390a542ef95211 (DEFAULT_TABLE table=test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:01.692706 12448 raft_consensus.cc:2258] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:01.692777 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbdad21d6e6478f88390a542ef95211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.694715 12450 ts_tablet_manager.cc:1918] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:01.694990 12449 tablet_replica.cc:330] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: stopping tablet replica
I20260807 22:50:01.695405 12449 raft_consensus.cc:2229] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:01.695812 12449 raft_consensus.cc:2258] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:01.696211 12450 log.cc:1196] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/8479143d22fe4232a63ba56c78f951fd
I20260807 22:50:01.696815 12450 ts_tablet_manager.cc:1939] T 8479143d22fe4232a63ba56c78f951fd P 42c1677b2ab9471e9bfe03fef0cca643: Deleting consensus metadata
I20260807 22:50:01.698774 11852 catalog_manager.cc:4841] TS 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371): tablet 8479143d22fe4232a63ba56c78f951fd (table test-table [id=181de52e8dc847439771c25705c22763]) successfully deleted
I20260807 22:50:01.699257 12309 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbdad21d6e6478f88390a542ef95211 (DEFAULT_TABLE table=test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:01.699695 12451 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Bootstrap starting.
I20260807 22:50:01.699922 12309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbdad21d6e6478f88390a542ef95211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:01.701274 12449 ts_tablet_manager.cc:1905] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:01.701555 12451 tablet_bootstrap.cc:654] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.708952 12448 ts_tablet_manager.cc:1905] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:01.709815 12451 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: No bootstrap required, opened a new log
I20260807 22:50:01.709954 12451 ts_tablet_manager.cc:1397] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260807 22:50:01.710701 12451 raft_consensus.cc:348] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.710943 12451 raft_consensus.cc:374] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.711026 12451 raft_consensus.cc:729] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.711272 12451 consensus_queue.cc:260] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.714093 12451 ts_tablet_manager.cc:1428] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:50:01.718328 12448 ts_tablet_manager.cc:1918] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:01.718562 12448 log.cc:1196] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/8479143d22fe4232a63ba56c78f951fd
I20260807 22:50:01.719380 12448 ts_tablet_manager.cc:1939] T 8479143d22fe4232a63ba56c78f951fd P 97970aff3cc649f1a31fc735739159ce: Deleting consensus metadata
I20260807 22:50:01.719857 12449 ts_tablet_manager.cc:1918] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:01.721261 12453 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Bootstrap starting.
I20260807 22:50:01.721411 11852 catalog_manager.cc:4841] TS 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405): tablet 8479143d22fe4232a63ba56c78f951fd (table test-table [id=181de52e8dc847439771c25705c22763]) successfully deleted
I20260807 22:50:01.720057 12449 log.cc:1196] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/8479143d22fe4232a63ba56c78f951fd
I20260807 22:50:01.722441 12449 ts_tablet_manager.cc:1939] T 8479143d22fe4232a63ba56c78f951fd P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting consensus metadata
I20260807 22:50:01.724363 11852 catalog_manager.cc:4841] TS 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363): tablet 8479143d22fe4232a63ba56c78f951fd (table test-table [id=181de52e8dc847439771c25705c22763]) successfully deleted
I20260807 22:50:01.724699 12454 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Bootstrap starting.
I20260807 22:50:01.725924 12453 tablet_bootstrap.cc:654] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.726552 12454 tablet_bootstrap.cc:654] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:01.728507 12453 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: No bootstrap required, opened a new log
I20260807 22:50:01.728679 12453 ts_tablet_manager.cc:1397] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 22:50:01.728839 12454 tablet_bootstrap.cc:492] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: No bootstrap required, opened a new log
I20260807 22:50:01.729024 12454 ts_tablet_manager.cc:1397] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260807 22:50:01.729568 12453 raft_consensus.cc:348] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.729805 12453 raft_consensus.cc:374] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.729934 12453 raft_consensus.cc:729] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.729862 12454 raft_consensus.cc:348] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.730120 12454 raft_consensus.cc:374] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:01.730219 12454 raft_consensus.cc:729] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Initialized, Role: FOLLOWER
I20260807 22:50:01.730242 12453 consensus_queue.cc:260] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.730477 12454 consensus_queue.cc:260] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:01.730937 12453 ts_tablet_manager.cc:1428] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:50:01.731142 12454 ts_tablet_manager.cc:1428] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:02.131906 12408 raft_consensus.cc:482] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:02.132282 12408 raft_consensus.cc:504] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:02.134433 12408 leader_election.cc:288] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:02.144454 12458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.134927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:42527 (local address 127.9.194.194:44371)
0807 22:50:02.136556 (+ 1629us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:02.136563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:02.136586 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:02.136671 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:02.136676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:02.136763 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:02.137554 (+ 791us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:02.137772 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.139178 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.141856 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.143216 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.143418 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.143549 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:02.143951 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 22:50:02.144099 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1473,"thread_start_us":1260,"threads_started":1}
I20260807 22:50:02.144551 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.134745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.9.194.193:42527)
0807 22:50:02.135732 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:02.135757 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:02.135933 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:02.136960 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:02.136967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:02.136999 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:02.137373 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.137389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.139324 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.139329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.141643 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.141657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.143479 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.143488 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.143685 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.143724 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:50:02.143906 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 22:50:02.144070 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":61,"thread_start_us":608,"threads_started":1}
I20260807 22:50:02.144979 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.135904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:38363 (local address 127.9.194.193:57357)
0807 22:50:02.136713 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:02.136730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:02.136892 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:02.137675 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:02.137680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:02.137704 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:02.138068 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.138076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.139194 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.139200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.142376 (+ 3176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.142389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.144072 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.144078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.144286 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.144327 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:50:02.144406 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:50:02.144559 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":120,"thread_start_us":700,"threads_started":1}
I20260807 22:50:02.145722 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.136453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:57357 (local address 127.9.194.195:38363)
0807 22:50:02.137414 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:02.137419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:02.137434 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:50:02.137488 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:02.137491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:02.137543 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:02.137658 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:02.138270 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.139051 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.142546 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.143782 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.145063 (+ 1281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.145152 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:02.145215 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 22:50:02.145360 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":703,"threads_started":1}
I20260807 22:50:02.145725 12188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bbdad21d6e6478f88390a542ef95211" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643" is_pre_election: true
I20260807 22:50:02.146193 12188 raft_consensus.cc:2454] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 0.
I20260807 22:50:02.146451 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bbdad21d6e6478f88390a542ef95211" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" is_pre_election: true
I20260807 22:50:02.146725 12331 raft_consensus.cc:2454] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 0.
I20260807 22:50:02.146898 11962 leader_election.cc:302] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:02.147361 12408 raft_consensus.cc:2781] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:02.147526 12408 raft_consensus.cc:482] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:02.147620 12408 raft_consensus.cc:3037] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:02.151134 12408 raft_consensus.cc:504] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:02.151863 12408 leader_election.cc:288] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 election: Requested vote from peers 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:02.152284 12188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bbdad21d6e6478f88390a542ef95211" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643"
I20260807 22:50:02.152534 12188 raft_consensus.cc:3037] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:02.152541 12331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bbdad21d6e6478f88390a542ef95211" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
I20260807 22:50:02.152791 12331 raft_consensus.cc:3037] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:02.154776 12188 raft_consensus.cc:2454] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 1.
I20260807 22:50:02.155277 11962 leader_election.cc:302] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:02.155576 12408 raft_consensus.cc:2781] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:02.155995 12408 raft_consensus.cc:686] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 1 LEADER]: Becoming Leader. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Running, Role: LEADER
I20260807 22:50:02.156476 12331 raft_consensus.cc:2454] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 1.
I20260807 22:50:02.156415 12408 consensus_queue.cc:237] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:02.160418 11866 catalog_manager.cc:5495] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 97970aff3cc649f1a31fc735739159ce (127.9.194.193). New cstate: current_term: 1 leader_uuid: "97970aff3cc649f1a31fc735739159ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:02.317404 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.310610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.9.194.193:38405)
0807 22:50:02.311226 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:02.311231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:02.311251 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:50:02.311381 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:02.311386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:02.311510 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:02.311713 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:02.312136 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.313219 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.314619 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.315283 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.315415 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.315887 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:02.316681 (+ 794us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:02.316820 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:02.316960 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 22:50:02.317045 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":330,"threads_started":1}
I20260807 22:50:02.334147 12458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.328632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37919 (local address 127.9.194.194:44371)
0807 22:50:02.328835 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:02.328843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:02.329260 (+ 417us) server_negotiation.cc:407] Connection header received
0807 22:50:02.329492 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:02.329496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:02.329563 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:02.329668 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:02.330181 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.330950 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.332433 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.332843 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.332975 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.333390 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:02.333489 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:02.333588 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:02.333732 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 22:50:02.333812 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:50:02.334230 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.328547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.9.194.193:37919)
0807 22:50:02.329150 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:02.329166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:02.329339 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:02.329688 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:02.329692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:02.329704 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:02.330010 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.330019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.331084 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.331091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.332277 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.332289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.333016 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.333024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.333159 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.333210 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:02.333590 (+ 380us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:02.333604 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:50:02.333710 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:50:02.333854 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":325,"threads_started":1}
I20260807 22:50:02.343740 12331 raft_consensus.cc:1261] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Refusing update from remote peer 97970aff3cc649f1a31fc735739159ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:02.343812 12188 raft_consensus.cc:1261] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Refusing update from remote peer 97970aff3cc649f1a31fc735739159ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:02.344528 12461 consensus_queue.cc:1035] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:02.345006 12408 consensus_queue.cc:1035] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:02.380836 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.375221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:42183 (local address 127.9.194.193:38405)
0807 22:50:02.375410 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:02.375417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:02.375791 (+ 374us) server_negotiation.cc:407] Connection header received
0807 22:50:02.375995 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:02.375999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:02.376060 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:02.376134 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:02.376591 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.377353 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.378964 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.379581 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.379734 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.380122 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:02.380225 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:02.380318 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:02.380459 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 22:50:02.380535 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:50:02.380924 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:02.375066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38405 (local address 127.9.194.194:42183)
0807 22:50:02.375696 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:02.375717 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:02.375866 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:02.376183 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:02.376188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:02.376201 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:02.376458 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.376466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.377493 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.377498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.378832 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:02.378844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:02.379771 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:02.379778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:02.379930 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:02.379967 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:02.380329 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:02.380342 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:50:02.380436 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 22:50:02.380563 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":372,"threads_started":1}
W20260807 22:50:02.382261 12465 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:50:03.982944 11866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:03.983158 11866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:03.987339 11866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Sending DeleteTablet for 3 replicas of tablet 4bbdad21d6e6478f88390a542ef95211
I20260807 22:50:03.988235 12013 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bbdad21d6e6478f88390a542ef95211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:03 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41692
I20260807 22:50:03.988315 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bbdad21d6e6478f88390a542ef95211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:03 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37820
I20260807 22:50:03.988525 12309 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bbdad21d6e6478f88390a542ef95211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:03 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:34172
I20260807 22:50:04.001226 12513 tablet_replica.cc:330] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: stopping tablet replica
I20260807 22:50:04.001556 12511 tablet_replica.cc:330] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: stopping tablet replica
I20260807 22:50:04.001593 12513 raft_consensus.cc:2229] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:04.001845 12511 raft_consensus.cc:2229] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:04.001928 12513 raft_consensus.cc:2258] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:04.002594 12511 raft_consensus.cc:2258] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:04.003363 12513 ts_tablet_manager.cc:1905] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.003834 12511 ts_tablet_manager.cc:1905] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.004813 12512 tablet_replica.cc:330] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: stopping tablet replica
I20260807 22:50:04.005010 12512 raft_consensus.cc:2229] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:04.005370 12512 raft_consensus.cc:2258] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:04.006457 12512 ts_tablet_manager.cc:1905] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.011114 12511 ts_tablet_manager.cc:1918] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:04.011294 12511 log.cc:1196] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/4bbdad21d6e6478f88390a542ef95211
I20260807 22:50:04.011759 12513 ts_tablet_manager.cc:1918] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:04.011917 12513 log.cc:1196] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/4bbdad21d6e6478f88390a542ef95211
I20260807 22:50:04.011826 12511 ts_tablet_manager.cc:1939] T 4bbdad21d6e6478f88390a542ef95211 P 97970aff3cc649f1a31fc735739159ce: Deleting consensus metadata
I20260807 22:50:04.012387 12513 ts_tablet_manager.cc:1939] T 4bbdad21d6e6478f88390a542ef95211 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting consensus metadata
I20260807 22:50:04.013406 12512 ts_tablet_manager.cc:1918] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:04.013381 11852 catalog_manager.cc:4841] TS 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405): tablet 4bbdad21d6e6478f88390a542ef95211 (table test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]) successfully deleted
I20260807 22:50:04.013581 12512 log.cc:1196] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/4bbdad21d6e6478f88390a542ef95211
I20260807 22:50:04.013701 11852 catalog_manager.cc:4841] TS 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363): tablet 4bbdad21d6e6478f88390a542ef95211 (table test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]) successfully deleted
I20260807 22:50:04.014062 12512 ts_tablet_manager.cc:1939] T 4bbdad21d6e6478f88390a542ef95211 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting consensus metadata
I20260807 22:50:04.014999 11852 catalog_manager.cc:4841] TS 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371): tablet 4bbdad21d6e6478f88390a542ef95211 (table test-table [id=c88d9bb9bc3940f5aa4f9402e8ca2fee]) successfully deleted
Aug 07 22:50:04 dist-test-slave-5k54 krb5kdc[11821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:50:04 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:04.055917 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.040004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54796 (local address 127.9.194.254:41419)
0807 22:50:04.040748 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.040755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.040781 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:50:04.041004 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.041009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.041073 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.041189 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:04.042559 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.043391 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.045238 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.046026 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.046198 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.050412 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:04.050417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:04.050420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:04.050450 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:04.053313 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:04.053993 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:04.053998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:04.054000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:04.054156 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:04.054499 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:04.054502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:04.054505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:04.054776 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:04.054958 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.055275 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.055515 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":363,"threads_started":1}
I20260807 22:50:04.059609 11866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:04.060217 11866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:04.067055 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 4146fd884ba3494da18855f5514c4339 (DEFAULT_TABLE table=test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.067335 12013 tablet_service.cc:1467] Processing CreateTablet for tablet 4146fd884ba3494da18855f5514c4339 (DEFAULT_TABLE table=test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.067577 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4146fd884ba3494da18855f5514c4339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.067832 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4146fd884ba3494da18855f5514c4339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.075212 12309 tablet_service.cc:1467] Processing CreateTablet for tablet 4146fd884ba3494da18855f5514c4339 (DEFAULT_TABLE table=test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.075855 12309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4146fd884ba3494da18855f5514c4339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.076413 12523 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Bootstrap starting.
I20260807 22:50:04.077914 12523 tablet_bootstrap.cc:654] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.079238 12524 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Bootstrap starting.
I20260807 22:50:04.081049 12524 tablet_bootstrap.cc:654] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.083076 12523 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: No bootstrap required, opened a new log
I20260807 22:50:04.083290 12523 ts_tablet_manager.cc:1397] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 22:50:04.083318 12524 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: No bootstrap required, opened a new log
I20260807 22:50:04.083314 12527 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Bootstrap starting.
I20260807 22:50:04.083477 12524 ts_tablet_manager.cc:1397] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260807 22:50:04.084318 12524 raft_consensus.cc:348] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.084321 12523 raft_consensus.cc:348] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.084591 12523 raft_consensus.cc:374] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.084591 12524 raft_consensus.cc:374] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.084692 12523 raft_consensus.cc:729] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.084693 12524 raft_consensus.cc:729] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.084949 12527 tablet_bootstrap.cc:654] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.084962 12524 consensus_queue.cc:260] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.085500 12523 consensus_queue.cc:260] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.086651 12523 ts_tablet_manager.cc:1428] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:04.087687 12524 ts_tablet_manager.cc:1428] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:50:04.089844 12527 tablet_bootstrap.cc:492] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: No bootstrap required, opened a new log
I20260807 22:50:04.090026 12527 ts_tablet_manager.cc:1397] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 22:50:04.090773 12527 raft_consensus.cc:348] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.090991 12527 raft_consensus.cc:374] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.091089 12527 raft_consensus.cc:729] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.091365 12527 consensus_queue.cc:260] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.092278 12527 ts_tablet_manager.cc:1428] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:50:04.102685 12530 raft_consensus.cc:482] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:04.103008 12530 raft_consensus.cc:504] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.104671 12530 leader_election.cc:288] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371)
I20260807 22:50:04.112437 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.105576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38405 (local address 127.9.194.195:48651)
0807 22:50:04.106332 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:04.106350 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:04.106503 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:04.107089 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:04.107092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:04.107114 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:04.107522 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.107538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.108929 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.108934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.110356 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.110369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.111611 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.111618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.111801 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.111825 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:50:04.111936 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:50:04.112072 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":467,"threads_started":1}
I20260807 22:50:04.112445 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.105625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:48651 (local address 127.9.194.193:38405)
0807 22:50:04.106348 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.106356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.106525 (+ 169us) server_negotiation.cc:407] Connection header received
0807 22:50:04.106642 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.106650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.106752 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.106872 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:04.107728 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.108793 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.110526 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.111340 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.111605 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.111684 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.111964 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.112097 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":377,"threads_started":1}
I20260807 22:50:04.113085 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4146fd884ba3494da18855f5514c4339" candidate_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce" is_pre_election: true
I20260807 22:50:04.113374 12040 raft_consensus.cc:2454] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229c45515b934dd4af45b0ea1a4bd4a9 in term 0.
I20260807 22:50:04.113413 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.106515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.9.194.195:33365)
0807 22:50:04.107271 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:04.107295 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:04.107531 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:04.108465 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:04.108473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:04.108500 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:04.108852 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.108860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.109959 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.109967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.111293 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.111304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.112585 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.112592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.112818 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.112847 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 22:50:04.112929 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:50:04.113053 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":323,"threads_started":1}
I20260807 22:50:04.113514 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.106652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:33365 (local address 127.9.194.194:44371)
0807 22:50:04.108116 (+ 1464us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.108120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.108140 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:50:04.108229 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.108233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.108319 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.108426 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:04.109016 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.109808 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.111500 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.112365 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.112556 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.112627 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.113026 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.113169 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1347,"thread_start_us":1181,"threads_started":1}
I20260807 22:50:04.113999 12256 leader_election.cc:302] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229c45515b934dd4af45b0ea1a4bd4a9, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:04.114111 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4146fd884ba3494da18855f5514c4339" candidate_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643" is_pre_election: true
I20260807 22:50:04.114418 12530 raft_consensus.cc:2781] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:04.114413 12187 raft_consensus.cc:2454] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229c45515b934dd4af45b0ea1a4bd4a9 in term 0.
I20260807 22:50:04.114535 12530 raft_consensus.cc:482] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:04.114647 12530 raft_consensus.cc:3037] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.117102 12530 raft_consensus.cc:504] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.117787 12530 leader_election.cc:288] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [CANDIDATE]: Term 1 election: Requested vote from peers 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405), 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371)
I20260807 22:50:04.118249 12040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4146fd884ba3494da18855f5514c4339" candidate_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97970aff3cc649f1a31fc735739159ce"
I20260807 22:50:04.118359 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4146fd884ba3494da18855f5514c4339" candidate_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643"
I20260807 22:50:04.118535 12040 raft_consensus.cc:3037] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.118582 12187 raft_consensus.cc:3037] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.120922 12040 raft_consensus.cc:2454] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229c45515b934dd4af45b0ea1a4bd4a9 in term 1.
I20260807 22:50:04.121165 12187 raft_consensus.cc:2454] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229c45515b934dd4af45b0ea1a4bd4a9 in term 1.
I20260807 22:50:04.121356 12256 leader_election.cc:302] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229c45515b934dd4af45b0ea1a4bd4a9, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:04.121699 12530 raft_consensus.cc:2781] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:04.122184 12530 raft_consensus.cc:686] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 LEADER]: Becoming Leader. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Running, Role: LEADER
I20260807 22:50:04.122597 12530 consensus_queue.cc:237] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.126060 11868 catalog_manager.cc:5495] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195). New cstate: current_term: 1 leader_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } health_report { overall_health: HEALTHY } } }
I20260807 22:50:04.149988 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.144468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.9.194.195:38363)
0807 22:50:04.145107 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.145112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.145131 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:04.145205 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.145208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.145311 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.145448 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:04.145853 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.146618 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.147953 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.148342 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.148435 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.148853 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.149472 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.149572 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.149647 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.149704 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":372,"threads_started":1}
I20260807 22:50:04.165467 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.160003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:44183 (local address 127.9.194.194:44371)
0807 22:50:04.160220 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.160229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.160656 (+ 427us) server_negotiation.cc:407] Connection header received
0807 22:50:04.160882 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.160886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.160959 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.161053 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:04.161578 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.162290 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.163722 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.164182 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.164313 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.164743 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.164851 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.164955 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.165091 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.165169 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:50:04.165597 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.159860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44371 (local address 127.9.194.195:44183)
0807 22:50:04.160552 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:04.160566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:04.160743 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:04.161067 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:04.161072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:04.161083 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:04.161397 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.161405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.162424 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.162437 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.163582 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.163593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.164376 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.164383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.164520 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.164568 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.164955 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.164969 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:50:04.165070 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:50:04.165193 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":370,"threads_started":1}
I20260807 22:50:04.174108 12040 raft_consensus.cc:1261] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Refusing update from remote peer 229c45515b934dd4af45b0ea1a4bd4a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:04.174227 12187 raft_consensus.cc:1261] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Refusing update from remote peer 229c45515b934dd4af45b0ea1a4bd4a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:04.174823 12530 consensus_queue.cc:1035] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:04.175141 12535 consensus_queue.cc:1035] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:04.181282 12490 mvcc.cc:205] Tried to move back new op lower bound from 7316041745091104768 to 7316041744890019840. Current Snapshot: MvccSnapshot[applied={T|T < 7316041745091104768}]
I20260807 22:50:04.204227 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.198206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59003 (local address 127.9.194.195:38363)
0807 22:50:04.198376 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.198388 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.198764 (+ 376us) server_negotiation.cc:407] Connection header received
0807 22:50:04.198903 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.198906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.198966 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.199036 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:04.199506 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.200502 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.202633 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.203087 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.203250 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.203632 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.203730 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.203838 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.203899 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.203960 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:50:04.204285 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.198085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:38363 (local address 127.9.194.194:59003)
0807 22:50:04.198670 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:04.198691 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:04.198806 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:04.199065 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:04.199068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:04.199078 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:04.199322 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.199330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.200713 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.200721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.202503 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:04.202516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.203274 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.203279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:04.203397 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.203480 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.203818 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.203826 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:50:04.203883 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:50:04.204006 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":355,"threads_started":1}
I20260807 22:50:04.238178 11868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:04.238488 11868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:04.241631 11868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Sending DeleteTablet for 3 replicas of tablet 4146fd884ba3494da18855f5514c4339
I20260807 22:50:04.242420 12013 tablet_service.cc:1514] Processing DeleteTablet for tablet 4146fd884ba3494da18855f5514c4339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:04 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41692
I20260807 22:50:04.242612 12309 tablet_service.cc:1514] Processing DeleteTablet for tablet 4146fd884ba3494da18855f5514c4339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:04 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:34172
I20260807 22:50:04.242729 12511 tablet_replica.cc:330] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: stopping tablet replica
I20260807 22:50:04.242923 12513 tablet_replica.cc:330] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: stopping tablet replica
I20260807 22:50:04.242944 12511 raft_consensus.cc:2229] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:04.243134 12511 raft_consensus.cc:2258] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:04.243129 12513 raft_consensus.cc:2229] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:04.243659 12513 raft_consensus.cc:2258] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:04.244118 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet 4146fd884ba3494da18855f5514c4339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:04 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37820
I20260807 22:50:04.244431 12512 tablet_replica.cc:330] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: stopping tablet replica
I20260807 22:50:04.244274 11868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:50:04.244625 12512 raft_consensus.cc:2229] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:04.244813 12512 raft_consensus.cc:2258] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:50:04.244944 11868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:04.245028 12513 ts_tablet_manager.cc:1905] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.246228 12512 ts_tablet_manager.cc:1905] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.253074 12513 ts_tablet_manager.cc:1918] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:04.253259 12513 log.cc:1196] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/4146fd884ba3494da18855f5514c4339
I20260807 22:50:04.253715 12513 ts_tablet_manager.cc:1939] T 4146fd884ba3494da18855f5514c4339 P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting consensus metadata
I20260807 22:50:04.254567 12512 ts_tablet_manager.cc:1918] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:04.254899 11852 catalog_manager.cc:4841] TS 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363): tablet 4146fd884ba3494da18855f5514c4339 (table test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]) successfully deleted
I20260807 22:50:04.255013 12512 log.cc:1196] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/4146fd884ba3494da18855f5514c4339
I20260807 22:50:04.255529 12512 ts_tablet_manager.cc:1939] T 4146fd884ba3494da18855f5514c4339 P 42c1677b2ab9471e9bfe03fef0cca643: Deleting consensus metadata
I20260807 22:50:04.256755 12161 tablet_service.cc:1467] Processing CreateTablet for tablet a4137fed49d442fdb096e133e47e21cb (DEFAULT_TABLE table=test-table [id=faea4953ba894fd294dd81da5897d003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.256762 12309 tablet_service.cc:1467] Processing CreateTablet for tablet a4137fed49d442fdb096e133e47e21cb (DEFAULT_TABLE table=test-table [id=faea4953ba894fd294dd81da5897d003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.256868 12013 tablet_service.cc:1467] Processing CreateTablet for tablet a4137fed49d442fdb096e133e47e21cb (DEFAULT_TABLE table=test-table [id=faea4953ba894fd294dd81da5897d003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:04.257162 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4137fed49d442fdb096e133e47e21cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.257169 12309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4137fed49d442fdb096e133e47e21cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.257313 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4137fed49d442fdb096e133e47e21cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:04.263608 12527 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Bootstrap starting.
I20260807 22:50:04.263854 12523 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Bootstrap starting.
I20260807 22:50:04.264266 12524 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Bootstrap starting.
I20260807 22:50:04.265324 12527 tablet_bootstrap.cc:654] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.265543 12523 tablet_bootstrap.cc:654] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.265743 12524 tablet_bootstrap.cc:654] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:04.267741 12527 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: No bootstrap required, opened a new log
I20260807 22:50:04.267922 12527 ts_tablet_manager.cc:1397] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260807 22:50:04.268697 12527 raft_consensus.cc:348] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.268965 12523 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: No bootstrap required, opened a new log
I20260807 22:50:04.268951 12527 raft_consensus.cc:374] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.269047 12527 raft_consensus.cc:729] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229c45515b934dd4af45b0ea1a4bd4a9, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.269122 12523 ts_tablet_manager.cc:1397] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 22:50:04.269354 12527 consensus_queue.cc:260] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.269858 12523 raft_consensus.cc:348] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.270025 12527 ts_tablet_manager.cc:1428] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:04.270136 12523 raft_consensus.cc:374] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.270215 12523 raft_consensus.cc:729] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.270464 12523 consensus_queue.cc:260] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.271085 12523 ts_tablet_manager.cc:1428] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:04.276863 12524 tablet_bootstrap.cc:492] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: No bootstrap required, opened a new log
I20260807 22:50:04.276867 11852 catalog_manager.cc:4841] TS 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371): tablet 4146fd884ba3494da18855f5514c4339 (table test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]) successfully deleted
I20260807 22:50:04.277029 12524 ts_tablet_manager.cc:1397] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260807 22:50:04.277876 12511 ts_tablet_manager.cc:1905] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:04.277799 12524 raft_consensus.cc:348] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.278062 12524 raft_consensus.cc:374] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:04.278149 12524 raft_consensus.cc:729] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c1677b2ab9471e9bfe03fef0cca643, State: Initialized, Role: FOLLOWER
I20260807 22:50:04.278393 12524 consensus_queue.cc:260] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.278888 12524 ts_tablet_manager.cc:1428] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 22:50:04.285144 12511 ts_tablet_manager.cc:1918] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:04.285362 12511 log.cc:1196] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/4146fd884ba3494da18855f5514c4339
I20260807 22:50:04.285892 12511 ts_tablet_manager.cc:1939] T 4146fd884ba3494da18855f5514c4339 P 97970aff3cc649f1a31fc735739159ce: Deleting consensus metadata
I20260807 22:50:04.287014 11852 catalog_manager.cc:4841] TS 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405): tablet 4146fd884ba3494da18855f5514c4339 (table test-table [id=b9bdfcbdc0814c5eac85e057bcfa4458]) successfully deleted
I20260807 22:50:04.329850 12461 raft_consensus.cc:482] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:04.330094 12461 raft_consensus.cc:504] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.330821 12461 leader_election.cc:288] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:04.331491 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4137fed49d442fdb096e133e47e21cb" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643" is_pre_election: true
I20260807 22:50:04.331583 12330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4137fed49d442fdb096e133e47e21cb" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" is_pre_election: true
I20260807 22:50:04.331804 12187 raft_consensus.cc:2454] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 0.
I20260807 22:50:04.331890 12330 raft_consensus.cc:2454] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 0.
I20260807 22:50:04.332388 11962 leader_election.cc:302] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229c45515b934dd4af45b0ea1a4bd4a9, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:04.332677 12461 raft_consensus.cc:2781] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:04.332787 12461 raft_consensus.cc:482] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:04.332904 12461 raft_consensus.cc:3037] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.335927 12461 raft_consensus.cc:504] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.336583 12461 leader_election.cc:288] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 election: Requested vote from peers 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371), 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363)
I20260807 22:50:04.337236 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4137fed49d442fdb096e133e47e21cb" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42c1677b2ab9471e9bfe03fef0cca643"
I20260807 22:50:04.337453 12187 raft_consensus.cc:3037] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.337342 12330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4137fed49d442fdb096e133e47e21cb" candidate_uuid: "97970aff3cc649f1a31fc735739159ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229c45515b934dd4af45b0ea1a4bd4a9"
I20260807 22:50:04.337585 12330 raft_consensus.cc:3037] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:04.340329 12330 raft_consensus.cc:2454] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 1.
I20260807 22:50:04.340535 12187 raft_consensus.cc:2454] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97970aff3cc649f1a31fc735739159ce in term 1.
I20260807 22:50:04.340986 11962 leader_election.cc:302] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42c1677b2ab9471e9bfe03fef0cca643, 97970aff3cc649f1a31fc735739159ce; no voters:
I20260807 22:50:04.341248 12461 raft_consensus.cc:2781] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:04.341470 12461 raft_consensus.cc:686] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 1 LEADER]: Becoming Leader. State: Replica: 97970aff3cc649f1a31fc735739159ce, State: Running, Role: LEADER
I20260807 22:50:04.341755 12461 consensus_queue.cc:237] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } }
I20260807 22:50:04.344619 11868 catalog_manager.cc:5495] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 97970aff3cc649f1a31fc735739159ce (127.9.194.193). New cstate: current_term: 1 leader_uuid: "97970aff3cc649f1a31fc735739159ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97970aff3cc649f1a31fc735739159ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:04.377167 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:04.371735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.9.194.193:38405)
0807 22:50:04.371898 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:04.371903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:04.371997 (+ 94us) server_negotiation.cc:407] Connection header received
0807 22:50:04.372149 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:04.372154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:04.372241 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:04.372338 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:04.372671 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.373774 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.375114 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:04.375808 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:04.375947 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:04.376396 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:04.376528 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:04.376618 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:04.376788 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:50:04.376878 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:50:04.386839 12187 raft_consensus.cc:1261] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Refusing update from remote peer 97970aff3cc649f1a31fc735739159ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:04.386931 12330 raft_consensus.cc:1261] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Refusing update from remote peer 97970aff3cc649f1a31fc735739159ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:04.387511 12461 consensus_queue.cc:1035] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "229c45515b934dd4af45b0ea1a4bd4a9" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:04.387812 12499 consensus_queue.cc:1035] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "42c1677b2ab9471e9bfe03fef0cca643" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:50:04.410856 12491 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:50:06.031684 11866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:06.031948 11866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:06.036134 11866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a07f18a29dab4fbfaf1f608107cea281: Sending DeleteTablet for 3 replicas of tablet a4137fed49d442fdb096e133e47e21cb
I20260807 22:50:06.036927 12013 tablet_service.cc:1514] Processing DeleteTablet for tablet a4137fed49d442fdb096e133e47e21cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:06 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41692
I20260807 22:50:06.036969 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet a4137fed49d442fdb096e133e47e21cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:06 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37820
I20260807 22:50:06.037118 12309 tablet_service.cc:1514] Processing DeleteTablet for tablet a4137fed49d442fdb096e133e47e21cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:06 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:34172
I20260807 22:50:06.038185 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11944
I20260807 22:50:06.056486 12582 tablet_replica.cc:330] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: stopping tablet replica
I20260807 22:50:06.056691 12582 raft_consensus.cc:2229] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:06.056937 12582 raft_consensus.cc:2258] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:06.057886 12582 ts_tablet_manager.cc:1905] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:06.063848 12581 tablet_replica.cc:330] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: stopping tablet replica
I20260807 22:50:06.064073 12581 raft_consensus.cc:2229] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:06.064429 12581 raft_consensus.cc:2258] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:06.064909 12583 tablet_replica.cc:330] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: stopping tablet replica
I20260807 22:50:06.065094 12583 raft_consensus.cc:2229] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:06.065315 12583 raft_consensus.cc:2258] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:06.065932 12582 ts_tablet_manager.cc:1918] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:06.066087 12582 log.cc:1196] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/a4137fed49d442fdb096e133e47e21cb
I20260807 22:50:06.066352 12583 ts_tablet_manager.cc:1905] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:06.066613 12582 ts_tablet_manager.cc:1939] T a4137fed49d442fdb096e133e47e21cb P 42c1677b2ab9471e9bfe03fef0cca643: Deleting consensus metadata
I20260807 22:50:06.067637 11852 catalog_manager.cc:4841] TS 42c1677b2ab9471e9bfe03fef0cca643 (127.9.194.194:44371): tablet a4137fed49d442fdb096e133e47e21cb (table test-table [id=faea4953ba894fd294dd81da5897d003]) successfully deleted
I20260807 22:50:06.068101 12581 ts_tablet_manager.cc:1905] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:06.073412 12581 ts_tablet_manager.cc:1918] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:06.073567 12581 log.cc:1196] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/a4137fed49d442fdb096e133e47e21cb
I20260807 22:50:06.073719 12583 ts_tablet_manager.cc:1918] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:50:06.073884 12583 log.cc:1196] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/a4137fed49d442fdb096e133e47e21cb
I20260807 22:50:06.074129 12581 ts_tablet_manager.cc:1939] T a4137fed49d442fdb096e133e47e21cb P 97970aff3cc649f1a31fc735739159ce: Deleting consensus metadata
I20260807 22:50:06.074361 12583 ts_tablet_manager.cc:1939] T a4137fed49d442fdb096e133e47e21cb P 229c45515b934dd4af45b0ea1a4bd4a9: Deleting consensus metadata
I20260807 22:50:06.075484 11852 catalog_manager.cc:4841] TS 97970aff3cc649f1a31fc735739159ce (127.9.194.193:38405): tablet a4137fed49d442fdb096e133e47e21cb (table test-table [id=faea4953ba894fd294dd81da5897d003]) successfully deleted
I20260807 22:50:06.075801 11852 catalog_manager.cc:4841] TS 229c45515b934dd4af45b0ea1a4bd4a9 (127.9.194.195:38363): tablet a4137fed49d442fdb096e133e47e21cb (table test-table [id=faea4953ba894fd294dd81da5897d003]) successfully deleted
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:06.084590 12585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:06.039826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.9.194.193:38405)
0807 22:50:06.055463 (+ 15637us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:06.055472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:06.055507 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:50:06.055584 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:06.055592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:06.055707 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:06.055851 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:06.057910 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.059553 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.062019 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.062957 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.063092 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:06.077280 (+ 14188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:06.077302 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:06.077324 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:06.077370 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:06.081347 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.082326 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.082340 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.082342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.082486 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.082897 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.082902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.082906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.083286 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:06.083628 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:06.083944 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 22:50:06.084186 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15435,"thread_start_us":15247,"threads_started":1}
W20260807 22:50:06.085399 12083 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54698
W20260807 22:50:06.086130 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11944: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:06.086483 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11944
I20260807 22:50:06.117774 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12092
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:06.140177 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:06.118926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.9.194.194:44371)
0807 22:50:06.119587 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:06.119595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:06.119616 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:50:06.119692 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:06.119697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:06.119800 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:06.119941 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:06.121859 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.122805 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.124805 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.125296 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.125451 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:06.131045 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:06.131069 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:06.131096 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:06.131155 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:06.136149 (+ 4994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.137069 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.137083 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.137090 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.137284 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.137742 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.137749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.137753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.138283 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:06.138611 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:06.139651 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0807 22:50:06.139861 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":412,"threads_started":1}
W20260807 22:50:06.141105 12230 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36586
W20260807 22:50:06.141878 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12092: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:06.142083 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12092
I20260807 22:50:06.178563 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12239
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[11821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:06.194720 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:06.179393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.9.194.195:38363)
0807 22:50:06.179950 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:06.179955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:06.179973 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:50:06.180028 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:06.180031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:06.180089 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:06.180180 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:06.181624 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.182383 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.183909 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:06.184355 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:06.184457 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:06.189057 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:06.189072 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:06.189088 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:06.189126 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:06.192193 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.192851 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.192857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.192860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.193044 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:06.193444 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:06.193448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:06.193450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:06.193797 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:06.194049 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:06.194279 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 22:50:06.194449 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":253,"threads_started":1}
W20260807 22:50:06.195492 12377 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57628
W20260807 22:50:06.196170 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12239: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:06.196374 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12239
I20260807 22:50:06.221596 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 11834
I20260807 22:50:06.222940 11912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:06.352293 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 11834
2026-08-07T22:50:06Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9112 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[12591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[12591](info): set up 2 sockets
Aug 07 22:50:06 dist-test-slave-5k54 krb5kdc[12591](info): commencing operation
krb5kdc: starting...
W20260807 22:50:07.480340 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:07 dist-test-slave-5k54 krb5kdc[12591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:07.692214 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39223
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:08.024143 12607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:08.024444 12607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:08.024610 12607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:08.032006 12607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:08.032101 12607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:08.032194 12607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:08.032315 12607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:08.032399 12607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:08.042323 12607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:08.043709 12607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:08.045030 12607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:08.057516 12615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:08.057605 12613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:08.058118 12607 server_base.cc:1034] running on GCE node
W20260807 22:50:08.058276 12612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:08.059150 12607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:08.060603 12607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:08.061805 12607 hybrid_clock.cc:648] HybridClock initialized: now 1786143008061764 us; error 53 us; skew 500 ppm
I20260807 22:50:08.062166 12607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:08 dist-test-slave-5k54 krb5kdc[12591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:08.065773 12607 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:08.067577 12607 webserver.cc:460] Webserver started at http://127.9.194.254:44199/ using document root <none> and password file <none>
I20260807 22:50:08.068480 12607 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:08.068621 12607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:08.068991 12607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:08.073359 12607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "31fefa98c0334af1a00ae2210dcc16df"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "3089f7432c8a9746a6b95684175abe07"
server_key_iv: "daf457201cfdedfb61fc549c2c12a59f"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.074476 12607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "31fefa98c0334af1a00ae2210dcc16df"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "3089f7432c8a9746a6b95684175abe07"
server_key_iv: "daf457201cfdedfb61fc549c2c12a59f"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.081453 12607 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 22:50:08.086169 12623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:08.087955 12607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:08.088133 12607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "31fefa98c0334af1a00ae2210dcc16df"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "3089f7432c8a9746a6b95684175abe07"
server_key_iv: "daf457201cfdedfb61fc549c2c12a59f"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.088402 12607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:08.122731 12607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:08.127067 12607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:08.127369 12607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:08.145823 12607 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:39223
I20260807 22:50:08.145855 12676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:39223 every 8 connection(s)
I20260807 22:50:08.147198 12607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:08.151032 12678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:08.151985 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12607
I20260807 22:50:08.152225 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:08.153097 9995 external_mini_cluster.cc:1428] Setting key 1aa3dd6906a0bd6c8c937cae3d70942d
I20260807 22:50:08.165577 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df: Bootstrap starting.
Aug 07 22:50:08 dist-test-slave-5k54 krb5kdc[12591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:08.169184 12678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:08.170279 12678 log.cc:824] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:08.173540 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df: No bootstrap required, opened a new log
I20260807 22:50:08.177023 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:08.155190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.9.194.254:39223)
0807 22:50:08.155999 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:08.156013 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:08.156053 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:50:08.157631 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:08.157656 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:08.158033 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:08.158362 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:08.160190 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.161451 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.163279 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.163992 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.164113 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:08.168736 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:08.168744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:08.168757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:08.168804 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:08.172400 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:08.173072 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:08.173081 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:08.173088 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:08.173299 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:08.173699 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:08.173704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:08.173706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:08.174433 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:08.174730 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:08.175100 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 22:50:08.175636 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":406,"threads_started":1}
I20260807 22:50:08.182835 12678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } }
I20260807 22:50:08.183184 12678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:08.183321 12678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fefa98c0334af1a00ae2210dcc16df, State: Initialized, Role: FOLLOWER
I20260807 22:50:08.184078 12678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } }
I20260807 22:50:08.184371 12678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:08.184515 12678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:08.184702 12678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:08.188428 12678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } }
I20260807 22:50:08.189073 12678 leader_election.cc:302] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31fefa98c0334af1a00ae2210dcc16df; no voters:
I20260807 22:50:08.191309 12678 leader_election.cc:288] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:08.191475 12683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:08.192299 12683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [term 1 LEADER]: Becoming Leader. State: Replica: 31fefa98c0334af1a00ae2210dcc16df, State: Running, Role: LEADER
I20260807 22:50:08.192883 12683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } }
I20260807 22:50:08.193869 12678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:08.196533 12685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31fefa98c0334af1a00ae2210dcc16df. Latest consensus state: current_term: 1 leader_uuid: "31fefa98c0334af1a00ae2210dcc16df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } } }
I20260807 22:50:08.196475 12684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31fefa98c0334af1a00ae2210dcc16df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fefa98c0334af1a00ae2210dcc16df" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39223 } } }
I20260807 22:50:08.196926 12685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:08.196926 12684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:08.197764 12689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:08.210889 12689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:08.219844 12689 catalog_manager.cc:1347] Generated new cluster ID: 9a2bf39079c141c4965018641ed24e07
I20260807 22:50:08.219970 12689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:08.230623 12689 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:08.232084 12689 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:08.238335 12689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31fefa98c0334af1a00ae2210dcc16df: Generated new TSK 0
I20260807 22:50:08.239418 12689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:08.303951 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:08.631125 12705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:08.631469 12705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:08.631672 12705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:08.638722 12705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:08.638867 12705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:08.639050 12705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:08.648571 12705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:08.649799 12705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:08.651111 12705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:08.664499 12710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:08.664543 12711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:08.664984 12713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:08.665701 12705 server_base.cc:1034] running on GCE node
I20260807 22:50:08.666324 12705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:08.667249 12705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:08.668619 12705 hybrid_clock.cc:648] HybridClock initialized: now 1786143008668554 us; error 50 us; skew 500 ppm
I20260807 22:50:08.668951 12705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:08 dist-test-slave-5k54 krb5kdc[12591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:08.672159 12705 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:08.673638 12705 webserver.cc:460] Webserver started at http://127.9.194.193:44863/ using document root <none> and password file <none>
I20260807 22:50:08.674341 12705 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:08.674492 12705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:08.674808 12705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:08.677729 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "e90243cfe8c94c68823c52c4c5b97b87"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "309a544e9559756f45b7014a0a890855"
server_key_iv: "7a22607f65714bfd862e947cf0879fc5"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.678540 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "e90243cfe8c94c68823c52c4c5b97b87"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "309a544e9559756f45b7014a0a890855"
server_key_iv: "7a22607f65714bfd862e947cf0879fc5"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.683666 12705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:50:08.687156 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:08.688527 12705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:08.688709 12705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "e90243cfe8c94c68823c52c4c5b97b87"
format_stamp: "Formatted at 2026-08-07 22:50:08 on dist-test-slave-5k54"
server_key: "309a544e9559756f45b7014a0a890855"
server_key_iv: "7a22607f65714bfd862e947cf0879fc5"
server_key_version: "encryptionkey@0"
I20260807 22:50:08.688908 12705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:08.720945 12705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:08.724870 12705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:08.725150 12705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:08.726114 12705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:08.727464 12705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:08.727564 12705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:08.727705 12705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:08.727790 12705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:08.761251 12705 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:46365
I20260807 22:50:08.761339 12845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:46365 every 8 connection(s)
I20260807 22:50:08.762538 12705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:08.766425 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12705
I20260807 22:50:08.766650 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:08.767364 9995 external_mini_cluster.cc:1428] Setting key 1ab07e64bf735f456f9d2b6020a3227f
Aug 07 22:50:08 dist-test-slave-5k54 krb5kdc[12591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:08.784185 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:08.765213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:33591 (local address 127.9.194.254:39223)
0807 22:50:08.765701 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:08.765708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:08.766757 (+ 1049us) server_negotiation.cc:407] Connection header received
0807 22:50:08.768804 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:08.768809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:08.768919 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:08.769034 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:08.770524 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.771277 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.772836 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.773252 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.773390 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:08.777210 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:08.777219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:08.777227 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:08.777271 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:08.780871 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:08.781585 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:08.781593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:08.781599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:08.781745 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:08.782166 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:08.782173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:08.782179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:08.782558 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:08.782727 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:08.783389 (+ 662us) server_negotiation.cc:299] Negotiation successful
0807 22:50:08.783679 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260807 22:50:08.784833 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:08.765725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39223 (local address 127.9.194.193:33591)
0807 22:50:08.766525 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:08.766577 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:08.768543 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:08.769294 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:08.769302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:08.769810 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:08.770346 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:08.770375 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.771469 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.771475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:08.772693 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:08.772705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:08.773495 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:08.773502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:08.773723 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:08.774403 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:08.774426 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:08.776974 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:08.781008 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:08.781014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:08.781025 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:08.781447 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:08.781858 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:08.781861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:08.781863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:08.782046 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:08.782674 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:08.782679 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:08.782990 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 22:50:08.783209 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 22:50:08.783653 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":352,"threads_started":1}
I20260807 22:50:08.786571 12846 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39223
I20260807 22:50:08.786921 12846 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:08.787796 12846 heartbeater.cc:511] Master 127.9.194.254:39223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
I20260807 22:50:08.789952 12641 ts_manager.cc:194] Registered new tserver with Master: e90243cfe8c94c68823c52c4c5b97b87 (127.9.194.193:46365)
I20260807 22:50:08.791399 12641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:33591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:08.818749 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:09.143635 12852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:09.143908 12852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:09.144075 12852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:09.151295 12852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:09.151399 12852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:09.151592 12852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:09.161155 12852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:09.162374 12852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:09.163697 12852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:09.176797 12858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:09.177017 12857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:09.177081 12860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:09.177769 12852 server_base.cc:1034] running on GCE node
I20260807 22:50:09.178413 12852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:09.179337 12852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:09.180568 12852 hybrid_clock.cc:648] HybridClock initialized: now 1786143009180520 us; error 42 us; skew 500 ppm
I20260807 22:50:09.180886 12852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:09 dist-test-slave-5k54 krb5kdc[12591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:09.184293 12852 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:09.185655 12852 webserver.cc:460] Webserver started at http://127.9.194.194:32931/ using document root <none> and password file <none>
I20260807 22:50:09.186322 12852 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:09.186475 12852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:09.186782 12852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:09.189671 12852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "2eb20eb9d2c341d6b3390261eb303221"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "ae2080a8262f870f13b7ae181c08c444"
server_key_iv: "1801fbbe060c1ae65448f2e2b7ca4fcc"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.190459 12852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "2eb20eb9d2c341d6b3390261eb303221"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "ae2080a8262f870f13b7ae181c08c444"
server_key_iv: "1801fbbe060c1ae65448f2e2b7ca4fcc"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.195940 12852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:50:09.199326 12867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.200760 12852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:50:09.200975 12852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "2eb20eb9d2c341d6b3390261eb303221"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "ae2080a8262f870f13b7ae181c08c444"
server_key_iv: "1801fbbe060c1ae65448f2e2b7ca4fcc"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.201181 12852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:09.229971 12852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:09.234297 12852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:09.234627 12852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:09.235661 12852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:09.237026 12852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:09.237131 12852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.237283 12852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:09.237370 12852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.272023 12852 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:43851
I20260807 22:50:09.272079 12990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:43851 every 8 connection(s)
I20260807 22:50:09.273386 12852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:09.278213 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12852
I20260807 22:50:09.278419 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:09.278985 9995 external_mini_cluster.cc:1428] Setting key 840aaa820c05ad25399d84323622ee6e
Aug 07 22:50:09 dist-test-slave-5k54 krb5kdc[12591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:09.298110 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:09.276150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:44587 (local address 127.9.194.254:39223)
0807 22:50:09.276338 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:09.276343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:09.277562 (+ 1219us) server_negotiation.cc:407] Connection header received
0807 22:50:09.280094 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:09.280098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:09.280175 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:09.280336 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:09.282205 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.282994 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.284873 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.285542 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.285681 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:09.289900 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:09.289910 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:09.289919 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:09.289969 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:09.294197 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:09.295119 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:09.295127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:09.295135 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:09.295314 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:09.295827 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:09.295844 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:09.295851 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:09.296321 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:09.296519 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:09.297276 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 22:50:09.297569 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:50:09.298784 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:09.276645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39223 (local address 127.9.194.194:44587)
0807 22:50:09.277381 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:09.277418 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:09.279781 (+ 2363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:09.280604 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:09.280612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:09.281397 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:09.282014 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:09.282033 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.283169 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.283175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:09.284685 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:09.284698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.285835 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.285845 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:09.286199 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:09.286976 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:09.287011 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:09.289628 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:09.294335 (+ 4707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:09.294343 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:09.294356 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:09.294948 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:09.295481 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:09.295485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:09.295487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:09.295690 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:09.296468 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:09.296473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:09.296807 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 22:50:09.297090 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 22:50:09.297508 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260807 22:50:09.300640 12991 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39223
I20260807 22:50:09.301034 12991 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:09.301954 12991 heartbeater.cc:511] Master 127.9.194.254:39223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:50:09.303685 12641 ts_manager.cc:194] Registered new tserver with Master: 2eb20eb9d2c341d6b3390261eb303221 (127.9.194.194:43851)
I20260807 22:50:09.304805 12641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:44587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:09.333127 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:09.657159 12996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:09.657419 12996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:09.657513 12996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:09.664559 12996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:09.664664 12996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:09.664774 12996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:09.674242 12996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:09.675542 12996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:09.676831 12996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:09.690006 13003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:09.690491 13005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:09.690717 13002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:09.691376 12996 server_base.cc:1034] running on GCE node
I20260807 22:50:09.692034 12996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:09.693168 12996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:09.694413 12996 hybrid_clock.cc:648] HybridClock initialized: now 1786143009694356 us; error 58 us; skew 500 ppm
I20260807 22:50:09.694739 12996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:09 dist-test-slave-5k54 krb5kdc[12591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:09.697945 12996 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:09.699363 12996 webserver.cc:460] Webserver started at http://127.9.194.195:34975/ using document root <none> and password file <none>
I20260807 22:50:09.700063 12996 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:09.700209 12996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:09.700527 12996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:09.703773 12996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "4dc724bc2bca40ccb05bbbce4ba34714"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "d11179b734a42a1cb5f4b70b7c31a696"
server_key_iv: "2c2b56ceb45cdeb37933c4eca2c2e25f"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.704623 12996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "4dc724bc2bca40ccb05bbbce4ba34714"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "d11179b734a42a1cb5f4b70b7c31a696"
server_key_iv: "2c2b56ceb45cdeb37933c4eca2c2e25f"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.709753 12996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:09.713171 13012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.714486 12996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:50:09.714610 12996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "4dc724bc2bca40ccb05bbbce4ba34714"
format_stamp: "Formatted at 2026-08-07 22:50:09 on dist-test-slave-5k54"
server_key: "d11179b734a42a1cb5f4b70b7c31a696"
server_key_iv: "2c2b56ceb45cdeb37933c4eca2c2e25f"
server_key_version: "encryptionkey@0"
I20260807 22:50:09.714774 12996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:09.731863 12996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:09.735819 12996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:09.736070 12996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:09.736964 12996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:09.738257 12996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:09.738329 12996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.738417 12996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:09.738476 12996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:09.773376 12996 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:37265
I20260807 22:50:09.773411 13137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:37265 every 8 connection(s)
I20260807 22:50:09.774648 12996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:09.783809 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 12996
I20260807 22:50:09.784013 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:09.784533 9995 external_mini_cluster.cc:1428] Setting key fb3b539d1e8e00369fde9d21561b8cbc
Aug 07 22:50:09 dist-test-slave-5k54 krb5kdc[12591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:09.794859 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:09.777266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:55143 (local address 127.9.194.254:39223)
0807 22:50:09.777464 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:09.777469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:09.778625 (+ 1156us) server_negotiation.cc:407] Connection header received
0807 22:50:09.780448 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:09.780452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:09.780564 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:09.780654 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:09.782063 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.782794 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.784478 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.785112 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.785211 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:09.789208 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:09.789213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:09.789216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:09.789248 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:09.791964 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:09.792615 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:09.792618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:09.792621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:09.792746 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:09.793211 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:09.793214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:09.793216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:09.793588 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:09.793728 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:09.794429 (+ 701us) server_negotiation.cc:299] Negotiation successful
0807 22:50:09.794596 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:50:09.795463 12846 heartbeater.cc:503] Master 127.9.194.254:39223 was elected leader, sending a full tablet report...
I20260807 22:50:09.796003 13140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:09.777775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39223 (local address 127.9.194.195:55143)
0807 22:50:09.778466 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:09.778503 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:09.780207 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:09.780849 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:09.780857 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:09.781362 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:09.781920 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:09.781936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.782928 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.782931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:09.784314 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:09.784325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:09.785345 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:09.785355 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:09.785635 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:09.786467 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:09.786507 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:09.789029 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:09.792094 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:09.792099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:09.792110 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:09.792507 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:09.792907 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:09.792912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:09.792915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:09.793105 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:09.793710 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:09.793716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:09.794018 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 22:50:09.794316 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 22:50:09.794672 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260807 22:50:09.797369 13138 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39223
I20260807 22:50:09.797680 13138 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:09.798512 13138 heartbeater.cc:511] Master 127.9.194.254:39223 requested a full tablet report, sending...
I20260807 22:50:09.800140 12640 ts_manager.cc:194] Registered new tserver with Master: 4dc724bc2bca40ccb05bbbce4ba34714 (127.9.194.195:37265)
I20260807 22:50:09.801019 12640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:55143
I20260807 22:50:09.812453 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:50:09.846673 12848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:09.844065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.9.194.254:39223)
0807 22:50:09.844286 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:09.844295 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:09.844679 (+ 384us) server_negotiation.cc:407] Connection header received
0807 22:50:09.844920 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:09.844924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:09.845007 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:09.845119 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:09.846480 (+ 1361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46714: BlockingRecv error: recv got EOF from 127.0.0.1:46714 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:50:09.850342 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12705
W20260807 22:50:09.853291 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12705: Not authorized: Client connection negotiation failed: client connection to 127.9.194.193:46365: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:50:09.853466 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12705
I20260807 22:50:09.870986 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12852
W20260807 22:50:09.873700 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12852: Not authorized: Client connection negotiation failed: client connection to 127.9.194.194:43851: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:50:09.873860 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12852
I20260807 22:50:09.890656 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12996
W20260807 22:50:09.893694 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12996: Not authorized: Client connection negotiation failed: client connection to 127.9.194.195:37265: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:50:09.893834 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12996
I20260807 22:50:09.911026 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 12607
I20260807 22:50:09.912428 12672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:10.015724 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 12607
2026-08-07T22:50:10Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3659 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:10 dist-test-slave-5k54 krb5kdc[13156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:10 dist-test-slave-5k54 krb5kdc[13156](info): set up 2 sockets
Aug 07 22:50:10 dist-test-slave-5k54 krb5kdc[13156](info): commencing operation
krb5kdc: starting...
W20260807 22:50:11.188481 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:11 dist-test-slave-5k54 krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:11.398732 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36375
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:46381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:11.725054 13171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:11.725340 13171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:11.725492 13171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:11.732893 13171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:11.732999 13171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:11.733088 13171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:11.733199 13171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:11.733309 13171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:11.743259 13171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:46381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:36375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:11.744616 13171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:11.745975 13171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:11.758469 13177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:11.758416 13176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:11.758464 13180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:11.759653 13171 server_base.cc:1034] running on GCE node
I20260807 22:50:11.760459 13171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:11.761791 13171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:11.763056 13171 hybrid_clock.cc:648] HybridClock initialized: now 1786143011763006 us; error 54 us; skew 500 ppm
I20260807 22:50:11.763514 13171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:11 dist-test-slave-5k54 krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:11.772172 13171 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:11.773923 13171 webserver.cc:460] Webserver started at http://127.9.194.254:34491/ using document root <none> and password file <none>
I20260807 22:50:11.774705 13171 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:11.774873 13171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:11.775225 13171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:11.778679 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "584de870ef984ad7bb83c6ffbf853b79"
format_stamp: "Formatted at 2026-08-07 22:50:11 on dist-test-slave-5k54"
server_key: "fa384a9876c2073eb9a92d62199ca3e7"
server_key_iv: "b417d4defef76e3aa10ecd07c6ca48df"
server_key_version: "encryptionkey@0"
I20260807 22:50:11.779634 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "584de870ef984ad7bb83c6ffbf853b79"
format_stamp: "Formatted at 2026-08-07 22:50:11 on dist-test-slave-5k54"
server_key: "fa384a9876c2073eb9a92d62199ca3e7"
server_key_iv: "b417d4defef76e3aa10ecd07c6ca48df"
server_key_version: "encryptionkey@0"
I20260807 22:50:11.785267 13171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:50:11.789045 13187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:11.790402 13171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:50:11.790601 13171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "584de870ef984ad7bb83c6ffbf853b79"
format_stamp: "Formatted at 2026-08-07 22:50:11 on dist-test-slave-5k54"
server_key: "fa384a9876c2073eb9a92d62199ca3e7"
server_key_iv: "b417d4defef76e3aa10ecd07c6ca48df"
server_key_version: "encryptionkey@0"
I20260807 22:50:11.790810 13171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:11.812893 13171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:11.817880 13171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:11.818198 13171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:11.838613 13171 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:36375
I20260807 22:50:11.838667 13242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:36375 every 8 connection(s)
I20260807 22:50:11.840171 13171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:11.843961 13243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:11.848271 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13171
I20260807 22:50:11.848455 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:11.849032 9995 external_mini_cluster.cc:1428] Setting key d01260b25ce82d149383074833b689cd
I20260807 22:50:11.858054 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79: Bootstrap starting.
I20260807 22:50:11.861809 13243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:11.862871 13243 log.cc:824] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:50:11 dist-test-slave-5k54 krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:11.865996 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79: No bootstrap required, opened a new log
I20260807 22:50:11.874552 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:11.851147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.9.194.254:36375)
0807 22:50:11.851971 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:11.851988 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:11.852050 (+ 62us) server_negotiation.cc:407] Connection header received
0807 22:50:11.853890 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:11.853920 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:11.854472 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:11.854920 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:11.856428 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:11.858406 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:11.859780 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:11.860830 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:11.861011 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:11.865779 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:11.865789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:11.865807 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:11.865854 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:11.869860 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:11.870600 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:11.870608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:11.870616 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:11.870834 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:11.871233 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:11.871237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:11.871240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:11.872027 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:11.872347 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:11.872739 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 22:50:11.873183 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":377,"threads_started":1}
I20260807 22:50:11.877046 13243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } }
I20260807 22:50:11.877463 13243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:11.877622 13243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584de870ef984ad7bb83c6ffbf853b79, State: Initialized, Role: FOLLOWER
I20260807 22:50:11.878257 13243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } }
I20260807 22:50:11.878556 13243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:11.878582 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:50:11.878710 13243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:11.878899 13243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:11.881102 13243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } }
I20260807 22:50:11.881579 13243 leader_election.cc:302] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 584de870ef984ad7bb83c6ffbf853b79; no voters:
I20260807 22:50:11.882267 13243 leader_election.cc:288] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:11.882375 13248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:11.884884 13243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:11.886548 13248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [term 1 LEADER]: Becoming Leader. State: Replica: 584de870ef984ad7bb83c6ffbf853b79, State: Running, Role: LEADER
I20260807 22:50:11.887246 13248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } }
I20260807 22:50:11.890471 13250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 584de870ef984ad7bb83c6ffbf853b79. Latest consensus state: current_term: 1 leader_uuid: "584de870ef984ad7bb83c6ffbf853b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } } }
I20260807 22:50:11.890504 13249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "584de870ef984ad7bb83c6ffbf853b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584de870ef984ad7bb83c6ffbf853b79" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36375 } } }
I20260807 22:50:11.890883 13250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:11.890883 13249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:11.893206 13260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:11.898505 13260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:11.908483 13260 catalog_manager.cc:1347] Generated new cluster ID: 02b0d3af58ea439d9984ce7a7f142e26
I20260807 22:50:11.908586 13260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:11.929435 13260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:11.930982 13260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:11.932512 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:11.921619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.9.194.254:36375)
0807 22:50:11.921781 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:11.921795 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:11.924012 (+ 2217us) server_negotiation.cc:407] Connection header received
0807 22:50:11.924184 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:11.924190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:11.924276 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:11.924403 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:11.925620 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:11.926739 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:11.928274 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:11.928952 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:11.929084 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:11.930452 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:11.930459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:11.930464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:11.931598 (+ 1134us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:11.931684 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:11.931720 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:11.931843 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:11.932016 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 22:50:11.932150 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:50:11.939792 13260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 584de870ef984ad7bb83c6ffbf853b79: Generated new TSK 0
I20260807 22:50:11.940776 13260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:50:11.953840 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13171
I20260807 22:50:11.955075 13238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:12.045724 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13171
2026-08-07T22:50:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2023 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:12 dist-test-slave-5k54 krb5kdc[13278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:12 dist-test-slave-5k54 krb5kdc[13278](info): set up 2 sockets
Aug 07 22:50:12 dist-test-slave-5k54 krb5kdc[13278](info): commencing operation
krb5kdc: starting...
W20260807 22:50:13.184561 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:13 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:13.401239 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33789
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:13.743068 13291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:13.743412 13291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:13.743616 13291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:13.750792 13291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:13.750936 13291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:13.751042 13291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:13.751132 13291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:13.751199 13291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:13.761034 13291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:13.762435 13291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:13.763844 13291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:13.775851 13297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:13.775954 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:13.775925 13300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:13.776691 13291 server_base.cc:1034] running on GCE node
I20260807 22:50:13.777413 13291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:13.778733 13291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:13.779975 13291 hybrid_clock.cc:648] HybridClock initialized: now 1786143013779949 us; error 47 us; skew 500 ppm
I20260807 22:50:13.780277 13291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:13 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:13.783339 13291 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:13.784904 13291 webserver.cc:460] Webserver started at http://127.9.194.254:40697/ using document root <none> and password file <none>
I20260807 22:50:13.785508 13291 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:13.785607 13291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:13.785879 13291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:13.788806 13291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "ebb3967f69284b98b4289a31814d2c7b"
format_stamp: "Formatted at 2026-08-07 22:50:13 on dist-test-slave-5k54"
server_key: "9f717241141202b1e72a626fce0ebbb4"
server_key_iv: "317e8e4e824b7b10f517d0c4691360e5"
server_key_version: "encryptionkey@0"
I20260807 22:50:13.789511 13291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "ebb3967f69284b98b4289a31814d2c7b"
format_stamp: "Formatted at 2026-08-07 22:50:13 on dist-test-slave-5k54"
server_key: "9f717241141202b1e72a626fce0ebbb4"
server_key_iv: "317e8e4e824b7b10f517d0c4691360e5"
server_key_version: "encryptionkey@0"
I20260807 22:50:13.794515 13291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 22:50:13.797941 13308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:13.799252 13291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:13.799506 13291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "ebb3967f69284b98b4289a31814d2c7b"
format_stamp: "Formatted at 2026-08-07 22:50:13 on dist-test-slave-5k54"
server_key: "9f717241141202b1e72a626fce0ebbb4"
server_key_iv: "317e8e4e824b7b10f517d0c4691360e5"
server_key_version: "encryptionkey@0"
I20260807 22:50:13.799692 13291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:13.815245 13291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:13.819240 13291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:13.819541 13291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:13.838419 13291 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33789
I20260807 22:50:13.838451 13363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33789 every 8 connection(s)
I20260807 22:50:13.839852 13291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:13.841420 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13291
I20260807 22:50:13.841671 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:13.842550 9995 external_mini_cluster.cc:1428] Setting key b55b586b3e38289bcd004845e424919e
I20260807 22:50:13.844815 13364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:50:13 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:13.858307 13364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b: Bootstrap starting.
I20260807 22:50:13.861843 13364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:13.862864 13364 log.cc:824] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:13.865690 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:13.844720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.9.194.254:33789)
0807 22:50:13.845581 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:13.845593 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:13.845633 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:50:13.847199 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:13.847221 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:13.847659 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:13.847996 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:13.849770 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:13.851145 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:13.852946 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:13.853664 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:13.853827 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:13.858531 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:13.858539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:13.858553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:13.858592 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:13.861755 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:13.862472 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:13.862479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:13.862485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:13.862649 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:13.863023 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:13.863030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:13.863033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:13.863750 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:13.864107 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:13.864374 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 22:50:13.864717 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":426,"threads_started":1}
I20260807 22:50:13.865978 13364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b: No bootstrap required, opened a new log
I20260807 22:50:13.873469 13364 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } }
I20260807 22:50:13.873836 13364 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:13.873950 13364 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebb3967f69284b98b4289a31814d2c7b, State: Initialized, Role: FOLLOWER
I20260807 22:50:13.874537 13364 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } }
I20260807 22:50:13.874778 13364 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:13.874927 13364 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:13.875103 13364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:13.877233 13364 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } }
I20260807 22:50:13.877708 13364 leader_election.cc:302] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebb3967f69284b98b4289a31814d2c7b; no voters:
I20260807 22:50:13.878258 13364 leader_election.cc:288] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:13.878391 13369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:13.879442 13369 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [term 1 LEADER]: Becoming Leader. State: Replica: ebb3967f69284b98b4289a31814d2c7b, State: Running, Role: LEADER
I20260807 22:50:13.880054 13369 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } }
I20260807 22:50:13.880501 13364 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:13.882872 13371 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebb3967f69284b98b4289a31814d2c7b. Latest consensus state: current_term: 1 leader_uuid: "ebb3967f69284b98b4289a31814d2c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } } }
I20260807 22:50:13.882789 13370 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebb3967f69284b98b4289a31814d2c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebb3967f69284b98b4289a31814d2c7b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33789 } } }
I20260807 22:50:13.883183 13370 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:13.883188 13371 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:13.884217 13378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:13.888810 13378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:13.897655 13378 catalog_manager.cc:1347] Generated new cluster ID: 45dff59d247e4af894f4915dc423bcc6
I20260807 22:50:13.897770 13378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:13.918735 13378 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:13.920115 13378 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:13.927316 13378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebb3967f69284b98b4289a31814d2c7b: Generated new TSK 0
I20260807 22:50:13.928169 13378 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:13.983266 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:14.314175 13392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:14.314448 13392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:14.314599 13392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:14.321714 13392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:14.321830 13392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:14.322033 13392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:14.331619 13392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:14.332966 13392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:14.334313 13392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:14.347164 13397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:14.347208 13400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:14.347229 13398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:14.348085 13392 server_base.cc:1034] running on GCE node
I20260807 22:50:14.348730 13392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:14.349650 13392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:14.350888 13392 hybrid_clock.cc:648] HybridClock initialized: now 1786143014350835 us; error 58 us; skew 500 ppm
I20260807 22:50:14.351205 13392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:14 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:14.354415 13392 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:14.355878 13392 webserver.cc:460] Webserver started at http://127.9.194.193:35841/ using document root <none> and password file <none>
I20260807 22:50:14.356559 13392 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:14.356702 13392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:14.356998 13392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:14.359956 13392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "0cb52e8978e74dacbf1b02d118953616"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "304b9d92196dbd73aa5e1e2619444ebb"
server_key_iv: "aaab2f3cddb09e81238b5e86768720be"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.360750 13392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "0cb52e8978e74dacbf1b02d118953616"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "304b9d92196dbd73aa5e1e2619444ebb"
server_key_iv: "aaab2f3cddb09e81238b5e86768720be"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.366405 13392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:14.370213 13407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.371671 13392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:14.371848 13392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "0cb52e8978e74dacbf1b02d118953616"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "304b9d92196dbd73aa5e1e2619444ebb"
server_key_iv: "aaab2f3cddb09e81238b5e86768720be"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.372048 13392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:14.394423 13392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:14.398470 13392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:14.398772 13392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:14.399766 13392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:14.401108 13392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:14.401212 13392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.401362 13392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:14.401432 13392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.434231 13392 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:38115
I20260807 22:50:14.434278 13527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:38115 every 8 connection(s)
I20260807 22:50:14.435608 13392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:14.445210 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13392
I20260807 22:50:14.445425 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:14.446125 9995 external_mini_cluster.cc:1428] Setting key 1a61b7b8334797598074340c336e6491
Aug 07 22:50:14 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:14.456935 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:14.438122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:45107 (local address 127.9.194.254:33789)
0807 22:50:14.438619 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:14.438624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:14.439499 (+ 875us) server_negotiation.cc:407] Connection header received
0807 22:50:14.441375 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:14.441381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:14.441471 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:14.441599 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:14.443093 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.443939 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.445727 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.446145 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.446284 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:14.450035 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:14.450044 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:14.450052 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:14.450100 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:14.453743 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:14.454467 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:14.454474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:14.454480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:14.454633 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:14.455037 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:14.455044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:14.455050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:14.455468 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:14.455589 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:14.456516 (+ 927us) server_negotiation.cc:299] Negotiation successful
0807 22:50:14.456684 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":282,"threads_started":1}
I20260807 22:50:14.458626 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:14.438597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33789 (local address 127.9.194.193:45107)
0807 22:50:14.439300 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:14.439338 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:14.441084 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:14.441838 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:14.441846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:14.442360 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:14.442920 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:14.442941 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.444181 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.444187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:14.445573 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:14.445585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.446436 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.446443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:14.446744 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:14.447502 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:14.447529 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:14.449799 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:14.453905 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:14.453912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:14.453926 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:14.454353 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:14.454741 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:14.454744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:14.454747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:14.454922 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:14.455593 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:14.455604 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:14.456076 (+ 472us) client_negotiation.cc:769] Sending connection context
0807 22:50:14.456421 (+ 345us) client_negotiation.cc:240] Negotiation successful
0807 22:50:14.457034 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260807 22:50:14.460225 13528 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33789
I20260807 22:50:14.460529 13528 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:14.461253 13528 heartbeater.cc:511] Master 127.9.194.254:33789 requested a full tablet report, sending...
I20260807 22:50:14.463299 13326 ts_manager.cc:194] Registered new tserver with Master: 0cb52e8978e74dacbf1b02d118953616 (127.9.194.193:38115)
I20260807 22:50:14.464766 13326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:45107
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:14.496295 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:14.819782 13534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:14.820143 13534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:14.820308 13534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:14.827636 13534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:14.827792 13534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:14.827978 13534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:14.837955 13534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:14.839675 13534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:14.841359 13534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:14.856539 13539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:14.856675 13543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:14.856586 13540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:14.858497 13534 server_base.cc:1034] running on GCE node
I20260807 22:50:14.859148 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:14.860195 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:14.861428 13534 hybrid_clock.cc:648] HybridClock initialized: now 1786143014861377 us; error 45 us; skew 500 ppm
I20260807 22:50:14.861752 13534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:14 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:14.865096 13534 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:14.866565 13534 webserver.cc:460] Webserver started at http://127.9.194.194:35165/ using document root <none> and password file <none>
I20260807 22:50:14.867267 13534 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:14.867411 13534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:14.867800 13534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:14.871797 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "3a495b9833514e49852fa8650d42db77"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "eabc8eeef6b8664cfe6489356407b74c"
server_key_iv: "454a0e4d6dea196f8bd673490732bf68"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.872938 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "3a495b9833514e49852fa8650d42db77"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "eabc8eeef6b8664cfe6489356407b74c"
server_key_iv: "454a0e4d6dea196f8bd673490732bf68"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.881166 13534 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260807 22:50:14.886268 13550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.888382 13534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:50:14.888566 13534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "3a495b9833514e49852fa8650d42db77"
format_stamp: "Formatted at 2026-08-07 22:50:14 on dist-test-slave-5k54"
server_key: "eabc8eeef6b8664cfe6489356407b74c"
server_key_iv: "454a0e4d6dea196f8bd673490732bf68"
server_key_version: "encryptionkey@0"
I20260807 22:50:14.888778 13534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:14.921175 13534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:14.925386 13534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:14.925632 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:14.926551 13534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:14.928037 13534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:14.928112 13534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.928207 13534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:14.928251 13534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:14.961094 13534 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:34431
I20260807 22:50:14.961140 13670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:34431 every 8 connection(s)
I20260807 22:50:14.962397 13534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:14.968102 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13534
I20260807 22:50:14.968295 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:14.968736 9995 external_mini_cluster.cc:1428] Setting key c096a4c4dc924c66d44ea31f4e2d9d66
Aug 07 22:50:14 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:14.983626 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:14.965061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:33621 (local address 127.9.194.254:33789)
0807 22:50:14.965510 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:14.965515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:14.966355 (+ 840us) server_negotiation.cc:407] Connection header received
0807 22:50:14.968270 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:14.968275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:14.968394 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:14.968517 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:14.970065 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.970805 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.972546 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.973296 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.973475 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:14.977294 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:14.977303 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:14.977311 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:14.977359 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:14.980189 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:14.981115 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:14.981119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:14.981121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:14.981286 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:14.981823 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:14.981828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:14.981831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:14.982111 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:14.982226 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:14.983100 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 22:50:14.983273 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":238,"threads_started":1}
I20260807 22:50:14.985159 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:14.965515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33789 (local address 127.9.194.194:33621)
0807 22:50:14.966199 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:14.966237 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:14.968003 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:14.968773 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:14.968781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:14.969326 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:14.969896 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:14.969913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.970959 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.970969 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:14.972381 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:14.972403 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:14.973500 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:14.973505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:14.973695 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:14.974392 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:14.974414 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:14.977043 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:14.980318 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:14.980329 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:14.980349 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:14.981004 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:14.981395 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:14.981403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:14.981409 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:14.981690 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:14.982234 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:14.982245 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:14.982707 (+ 462us) client_negotiation.cc:769] Sending connection context
0807 22:50:14.983015 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 22:50:14.983661 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":262,"threads_started":1}
I20260807 22:50:14.986699 13671 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33789
I20260807 22:50:14.987044 13671 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:14.987838 13671 heartbeater.cc:511] Master 127.9.194.254:33789 requested a full tablet report, sending...
I20260807 22:50:14.989226 13326 ts_manager.cc:194] Registered new tserver with Master: 3a495b9833514e49852fa8650d42db77 (127.9.194.194:34431)
I20260807 22:50:14.990085 13326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:33621
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:15.019186 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:15.348686 13677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:15.348990 13677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:15.349153 13677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:15.356674 13677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:15.356822 13677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:15.357010 13677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:15.367007 13677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:15.368435 13677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:15.369822 13677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:15.383574 13682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:15.383567 13683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:15.384052 13685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:15.385140 13677 server_base.cc:1034] running on GCE node
I20260807 22:50:15.385782 13677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:15.390924 13677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:15.392196 13677 hybrid_clock.cc:648] HybridClock initialized: now 1786143015392147 us; error 46 us; skew 500 ppm
I20260807 22:50:15.392575 13677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:15.396226 13677 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:15.397716 13677 webserver.cc:460] Webserver started at http://127.9.194.195:38513/ using document root <none> and password file <none>
I20260807 22:50:15.398403 13677 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:15.398507 13677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:15.398768 13677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:15.401804 13677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "9b2d1fbd264e4ccabcb16a259a212030"
format_stamp: "Formatted at 2026-08-07 22:50:15 on dist-test-slave-5k54"
server_key: "b3eafcb56550d54f2f3a2ebf91b0f84c"
server_key_iv: "ece7965538f77e987ca7af132a376b41"
server_key_version: "encryptionkey@0"
I20260807 22:50:15.402542 13677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "9b2d1fbd264e4ccabcb16a259a212030"
format_stamp: "Formatted at 2026-08-07 22:50:15 on dist-test-slave-5k54"
server_key: "b3eafcb56550d54f2f3a2ebf91b0f84c"
server_key_iv: "ece7965538f77e987ca7af132a376b41"
server_key_version: "encryptionkey@0"
I20260807 22:50:15.408206 13677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:50:15.411867 13693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:15.413345 13677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:50:15.413528 13677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "9b2d1fbd264e4ccabcb16a259a212030"
format_stamp: "Formatted at 2026-08-07 22:50:15 on dist-test-slave-5k54"
server_key: "b3eafcb56550d54f2f3a2ebf91b0f84c"
server_key_iv: "ece7965538f77e987ca7af132a376b41"
server_key_version: "encryptionkey@0"
I20260807 22:50:15.413726 13677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:15.428697 13677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:15.432873 13677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:15.433180 13677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:15.434157 13677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:15.435699 13677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:15.435797 13677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:15.435949 13677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:15.436021 13677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:15.467761 13528 heartbeater.cc:503] Master 127.9.194.254:33789 was elected leader, sending a full tablet report...
I20260807 22:50:15.470700 13677 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:35521
I20260807 22:50:15.470746 13813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:35521 every 8 connection(s)
I20260807 22:50:15.472110 13677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:15.479897 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13677
I20260807 22:50:15.480135 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:15.480645 9995 external_mini_cluster.cc:1428] Setting key 99c0d69f4f7aff6505100495bb9ad266
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:15.493010 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.474644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:46311 (local address 127.9.194.254:33789)
0807 22:50:15.474886 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:15.474893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:15.476257 (+ 1364us) server_negotiation.cc:407] Connection header received
0807 22:50:15.478131 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:15.478135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:15.478222 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:15.478325 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:15.480065 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.481051 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.483137 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.483682 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.483776 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.487206 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:15.487211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:15.487214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:15.487247 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:15.490109 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.490833 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.490838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.490840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.490957 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.491543 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.491546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.491548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.491814 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:15.491953 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:15.492597 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 22:50:15.492763 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:50:15.493935 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.475219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33789 (local address 127.9.194.195:46311)
0807 22:50:15.476114 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:15.476154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:15.477865 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:15.478519 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:15.478534 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:15.479096 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:15.479863 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:15.479892 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.481249 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.481253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:15.482998 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:15.483015 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.483943 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.483949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:15.484157 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.484763 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:15.484819 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:15.487024 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:15.490250 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:15.490257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:15.490269 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:15.490706 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:15.491056 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:15.491059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:15.491061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:15.491404 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:15.491931 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:15.491937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:15.492231 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 22:50:15.492449 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 22:50:15.492829 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":322,"threads_started":1}
I20260807 22:50:15.495638 13814 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33789
I20260807 22:50:15.496016 13814 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:15.496731 13814 heartbeater.cc:511] Master 127.9.194.254:33789 requested a full tablet report, sending...
I20260807 22:50:15.498099 13326 ts_manager.cc:194] Registered new tserver with Master: 9b2d1fbd264e4ccabcb16a259a212030 (127.9.194.195:35521)
I20260807 22:50:15.498849 13326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:46311
I20260807 22:50:15.508330 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:15.560958 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.547061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.9.194.254:33789)
0807 22:50:15.547317 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:15.547323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:15.547684 (+ 361us) server_negotiation.cc:407] Connection header received
0807 22:50:15.547939 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:15.547946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:15.548027 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:15.548131 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:15.549555 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.550341 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.551647 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.552060 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.552192 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.556107 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:15.556112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:15.556115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:15.556149 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:15.558532 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.559171 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.559175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.559177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.559333 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.559761 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.559765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.559767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.560098 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:15.560306 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:15.560533 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 22:50:15.560693 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 22:50:15.561486 13326 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39330
I20260807 22:50:15.563037 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13392
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:15.578743 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.564165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.9.194.193:38115)
0807 22:50:15.564766 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:15.564773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:15.564807 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:50:15.564864 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:15.564870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:15.565127 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:15.565266 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:15.566417 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.567463 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.568645 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.569206 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.569294 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.573192 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:15.573202 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:15.573216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:15.573248 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:15.576268 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.576944 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.576955 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.576958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.577083 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.577476 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.577479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.577480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.577793 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:15.578037 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:15.578272 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:50:15.578465 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":310,"threads_started":1}
W20260807 22:50:15.580053 13523 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55166
W20260807 22:50:15.581245 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13392: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:15.581465 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13392
I20260807 22:50:15.600025 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13534
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:15.617043 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.600873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.9.194.194:34431)
0807 22:50:15.601437 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:15.601443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:15.601469 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:50:15.601529 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:15.601535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:15.601756 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:15.601888 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:15.603325 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.604693 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.606097 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.606639 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.606746 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.611075 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:15.611085 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:15.611100 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:15.611140 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:15.614157 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.614855 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.614863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.614866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.615016 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.615354 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.615358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.615360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.615803 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:15.616068 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:15.616316 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 22:50:15.616537 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":290,"threads_started":1}
W20260807 22:50:15.618497 13666 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52506
W20260807 22:50:15.619766 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13534: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:15.619953 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13534
I20260807 22:50:15.638067 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13677
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:15.655190 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:15.638970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.9.194.195:35521)
0807 22:50:15.639582 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:15.639589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:15.639617 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:50:15.639674 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:15.639680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:15.639890 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:15.640006 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:15.641333 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.642400 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.643960 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:15.644547 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:15.644651 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:15.649390 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:15.649402 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:15.649418 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:15.649456 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:15.652610 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.653353 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.653360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.653363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.653493 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:15.653855 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:15.653859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:15.653861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:15.654204 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:15.654468 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:15.654684 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:50:15.654886 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":260,"threads_started":1}
W20260807 22:50:15.656586 13808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57080
W20260807 22:50:15.657691 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13677: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:50:15.657894 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13677
I20260807 22:50:15.675379 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13291
I20260807 22:50:15.676702 13359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:15.781894 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13291
2026-08-07T22:50:15Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3742 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13831](info): set up 2 sockets
Aug 07 22:50:15 dist-test-slave-5k54 krb5kdc[13831](info): commencing operation
krb5kdc: starting...
W20260807 22:50:16.952705 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:17 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:17.170706 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:42335
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:42335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:17.492923 13845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:17.493196 13845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:17.493284 13845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:17.500301 13845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:17.500403 13845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:17.500448 13845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:17.500505 13845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:17.500550 13845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:17.510188 13845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:42335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:42335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:17.511497 13845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:17.512784 13845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:17.526607 13851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:17.526607 13850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:17.526675 13853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:17.527729 13845 server_base.cc:1034] running on GCE node
I20260807 22:50:17.528930 13845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:17.530463 13845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:17.531710 13845 hybrid_clock.cc:648] HybridClock initialized: now 1786143017531656 us; error 52 us; skew 500 ppm
I20260807 22:50:17.532091 13845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:17 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:17.539835 13845 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:17.541334 13845 webserver.cc:460] Webserver started at http://127.9.194.254:36497/ using document root <none> and password file <none>
I20260807 22:50:17.542027 13845 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:17.542138 13845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:17.542438 13845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:17.545708 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "4d471081d807488ab0cc1b94c0e0f6ff"
format_stamp: "Formatted at 2026-08-07 22:50:17 on dist-test-slave-5k54"
server_key: "6252654741a03421cef55c18811bf78c"
server_key_iv: "9878a4758601a35c1dfa08e635f2fce0"
server_key_version: "encryptionkey@0"
I20260807 22:50:17.546525 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "4d471081d807488ab0cc1b94c0e0f6ff"
format_stamp: "Formatted at 2026-08-07 22:50:17 on dist-test-slave-5k54"
server_key: "6252654741a03421cef55c18811bf78c"
server_key_iv: "9878a4758601a35c1dfa08e635f2fce0"
server_key_version: "encryptionkey@0"
I20260807 22:50:17.552078 13845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 22:50:17.555717 13860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:17.557065 13845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:50:17.557260 13845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "4d471081d807488ab0cc1b94c0e0f6ff"
format_stamp: "Formatted at 2026-08-07 22:50:17 on dist-test-slave-5k54"
server_key: "6252654741a03421cef55c18811bf78c"
server_key_iv: "9878a4758601a35c1dfa08e635f2fce0"
server_key_version: "encryptionkey@0"
I20260807 22:50:17.557475 13845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:17.592126 13845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:17.596693 13845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:17.597010 13845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:17.615466 13845 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:42335
I20260807 22:50:17.615509 13916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:42335 every 8 connection(s)
I20260807 22:50:17.616852 13845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:17.619710 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13845
I20260807 22:50:17.619959 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:17.620631 9995 external_mini_cluster.cc:1428] Setting key 48784f6d6b8a1e0be4df7632ab31dda6
I20260807 22:50:17.620743 13917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:17.635890 13917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff: Bootstrap starting.
Aug 07 22:50:17 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:17.639285 13917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:17.640609 13917 log.cc:824] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:17.643505 13917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff: No bootstrap required, opened a new log
I20260807 22:50:17.644836 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:17.622591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.9.194.254:42335)
0807 22:50:17.623386 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:17.623404 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:17.623487 (+ 83us) server_negotiation.cc:407] Connection header received
0807 22:50:17.625426 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:17.625447 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:17.625824 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:17.626427 (+ 603us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:17.628434 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:17.629752 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:17.631572 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:17.632317 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:17.632483 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:17.637188 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:17.637197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:17.637214 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:17.637257 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:17.640769 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:17.641447 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:17.641455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:17.641463 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:17.641650 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:17.642052 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:17.642057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:17.642060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:17.642583 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:17.642878 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:17.643325 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 22:50:17.643843 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":361,"threads_started":1}
I20260807 22:50:17.651248 13917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } }
I20260807 22:50:17.651659 13917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:17.651762 13917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d471081d807488ab0cc1b94c0e0f6ff, State: Initialized, Role: FOLLOWER
I20260807 22:50:17.652293 13917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } }
I20260807 22:50:17.652524 13917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:17.652657 13917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:17.652833 13917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:17.654896 13917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } }
I20260807 22:50:17.655370 13917 leader_election.cc:302] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d471081d807488ab0cc1b94c0e0f6ff; no voters:
I20260807 22:50:17.656116 13917 leader_election.cc:288] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:17.656261 13923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:17.657344 13923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [term 1 LEADER]: Becoming Leader. State: Replica: 4d471081d807488ab0cc1b94c0e0f6ff, State: Running, Role: LEADER
I20260807 22:50:17.657823 13923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } }
I20260807 22:50:17.658224 13917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:17.660815 13924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } } }
I20260807 22:50:17.661600 13924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:17.661294 13925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d471081d807488ab0cc1b94c0e0f6ff. Latest consensus state: current_term: 1 leader_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d471081d807488ab0cc1b94c0e0f6ff" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 42335 } } }
I20260807 22:50:17.662065 13925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:17.674255 13930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:17.679579 13930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:17.687979 13930 catalog_manager.cc:1347] Generated new cluster ID: 7742e1ddb4514edeb63f28d2fab3ec66
I20260807 22:50:17.688097 13930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:17.714341 13930 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:17.716349 13930 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:17.727231 13930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d471081d807488ab0cc1b94c0e0f6ff: Generated new TSK 0
I20260807 22:50:17.728371 13930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:17.791328 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:18.117810 13947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:18.118108 13947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:18.118204 13947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:18.126179 13947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:18.126389 13947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:18.126602 13947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:18.136971 13947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:18.138422 13947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:18.139818 13947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:18.152745 13953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:18.152855 13952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:18.152875 13955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:18.153386 13947 server_base.cc:1034] running on GCE node
I20260807 22:50:18.153970 13947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:18.154908 13947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:18.156148 13947 hybrid_clock.cc:648] HybridClock initialized: now 1786143018156089 us; error 49 us; skew 500 ppm
I20260807 22:50:18.156513 13947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:18 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143018, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:18.159848 13947 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:18.161340 13947 webserver.cc:460] Webserver started at http://127.9.194.193:42653/ using document root <none> and password file <none>
I20260807 22:50:18.162070 13947 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:18.162195 13947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:18.162505 13947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:18.165381 13947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "b05bba5c1fb64b4cb4d369bfd8e4a831"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "203f8503a2a1ceb72eb784b0b08ab989"
server_key_iv: "6b721d5eaf7501fcdd3c5b13908dcb37"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.166145 13947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "b05bba5c1fb64b4cb4d369bfd8e4a831"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "203f8503a2a1ceb72eb784b0b08ab989"
server_key_iv: "6b721d5eaf7501fcdd3c5b13908dcb37"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.171272 13947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:50:18.174691 13962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.176195 13947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:18.176371 13947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "b05bba5c1fb64b4cb4d369bfd8e4a831"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "203f8503a2a1ceb72eb784b0b08ab989"
server_key_iv: "6b721d5eaf7501fcdd3c5b13908dcb37"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.176568 13947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:18.189090 13947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:18.192888 13947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:18.193189 13947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:18.194088 13947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:18.195375 13947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:18.195499 13947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.195636 13947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:18.195717 13947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.231640 13947 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:36315
I20260807 22:50:18.231671 14079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:36315 every 8 connection(s)
I20260807 22:50:18.232935 13947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:18.240971 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 13947
I20260807 22:50:18.241189 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:18.241950 9995 external_mini_cluster.cc:1428] Setting key 0a15af29888be49d049dae9a9aa093a3
Aug 07 22:50:18 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143018, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:18.253017 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:18.235479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:38343 (local address 127.9.194.254:42335)
0807 22:50:18.235885 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:18.235890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:18.236966 (+ 1076us) server_negotiation.cc:407] Connection header received
0807 22:50:18.238781 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:18.238785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:18.238861 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:18.238961 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:18.240501 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.241628 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.243231 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.243719 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.243895 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:18.247487 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:18.247491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:18.247495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:18.247525 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:18.250009 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:18.250740 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:18.250747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:18.250750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:18.250873 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:18.251296 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:18.251302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:18.251305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:18.251618 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:18.251768 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:18.252436 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 22:50:18.252662 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":210,"threads_started":1}
I20260807 22:50:18.253970 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:18.236085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:42335 (local address 127.9.194.193:38343)
0807 22:50:18.236804 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:18.236847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:18.238529 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:18.239155 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:18.239163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:18.239782 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:18.240331 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:18.240348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.241775 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.241778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:18.243066 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:18.243079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.243978 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.243987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:18.244225 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:18.244894 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:18.244917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:18.247248 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:18.250166 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:18.250174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:18.250189 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:18.250624 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:18.250988 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:18.250994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:18.250997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:18.251185 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:18.251740 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:18.251748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:18.252075 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 22:50:18.252330 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 22:50:18.252825 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":303,"threads_started":1}
I20260807 22:50:18.255592 14080 heartbeater.cc:347] Connected to a master server at 127.9.194.254:42335
I20260807 22:50:18.255931 14080 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:18.256788 14080 heartbeater.cc:511] Master 127.9.194.254:42335 requested a full tablet report, sending...
I20260807 22:50:18.259130 13878 ts_manager.cc:194] Registered new tserver with Master: b05bba5c1fb64b4cb4d369bfd8e4a831 (127.9.194.193:36315)
I20260807 22:50:18.260756 13878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:38343
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:18.290771 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:18.615015 14087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:18.615341 14087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:18.615536 14087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:18.622783 14087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:18.622942 14087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:18.623121 14087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:18.632951 14087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:18.634218 14087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:18.635583 14087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:18.648707 14094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:18.648727 14095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:18.648838 14098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:18.649812 14087 server_base.cc:1034] running on GCE node
I20260807 22:50:18.650384 14087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:18.651288 14087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:18.652509 14087 hybrid_clock.cc:648] HybridClock initialized: now 1786143018652460 us; error 58 us; skew 500 ppm
I20260807 22:50:18.652812 14087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:18 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143018, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:18.655908 14087 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:18.657296 14087 webserver.cc:460] Webserver started at http://127.9.194.194:41789/ using document root <none> and password file <none>
I20260807 22:50:18.657958 14087 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:18.658113 14087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:18.658419 14087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:18.661558 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "6d336a0257c3479c9bd2ce5ad5c39412"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "42cea2e786da0b0ad96329aaa9318345"
server_key_iv: "c26e5283f65a96b9f1c0ec5c54b7f5e0"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.662407 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "6d336a0257c3479c9bd2ce5ad5c39412"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "42cea2e786da0b0ad96329aaa9318345"
server_key_iv: "c26e5283f65a96b9f1c0ec5c54b7f5e0"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.667408 14087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260807 22:50:18.670801 14106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.672245 14087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:50:18.672405 14087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "6d336a0257c3479c9bd2ce5ad5c39412"
format_stamp: "Formatted at 2026-08-07 22:50:18 on dist-test-slave-5k54"
server_key: "42cea2e786da0b0ad96329aaa9318345"
server_key_iv: "c26e5283f65a96b9f1c0ec5c54b7f5e0"
server_key_version: "encryptionkey@0"
I20260807 22:50:18.672613 14087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:18.689154 14087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:18.692986 14087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:18.693264 14087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:18.694192 14087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:18.695524 14087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:18.695650 14087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.695796 14087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:18.695860 14087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:18.728464 14087 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:33827
I20260807 22:50:18.728513 14226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:33827 every 8 connection(s)
I20260807 22:50:18.729750 14087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:18.730283 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14087
I20260807 22:50:18.730499 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:18.731055 9995 external_mini_cluster.cc:1428] Setting key 68e488cdacf02120f3490380831ba96f
Aug 07 22:50:18 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143018, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:18.752564 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:18.732945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:57573 (local address 127.9.194.254:42335)
0807 22:50:18.733150 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:18.733158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:18.734690 (+ 1532us) server_negotiation.cc:407] Connection header received
0807 22:50:18.736769 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:18.736775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:18.736880 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:18.736994 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:18.738587 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.739656 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.741529 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.741952 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.742092 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:18.746062 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:18.746067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:18.746070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:18.746103 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:18.748824 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:18.749928 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:18.749933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:18.749935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:18.750062 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:18.750597 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:18.750600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:18.750602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:18.750989 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:18.751114 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:18.752157 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0807 22:50:18.752324 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:50:18.754371 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:18.733615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:42335 (local address 127.9.194.194:57573)
0807 22:50:18.734499 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:18.734559 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:18.736489 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:18.737246 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:18.737255 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:18.737808 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:18.738410 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:18.738429 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.739829 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.739838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:18.741364 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:18.741376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:18.742190 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:18.742198 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:18.742425 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:18.743111 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:18.743136 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:18.745813 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:18.748967 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:18.748980 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:18.749001 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:18.749733 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:18.750185 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:18.750194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:18.750201 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:18.750479 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:18.751140 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:18.751152 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:18.751706 (+ 554us) client_negotiation.cc:769] Sending connection context
0807 22:50:18.752099 (+ 393us) client_negotiation.cc:240] Negotiation successful
0807 22:50:18.752687 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":349,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:50:18.756027 14227 heartbeater.cc:347] Connected to a master server at 127.9.194.254:42335
I20260807 22:50:18.756381 14227 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:18.757228 14227 heartbeater.cc:511] Master 127.9.194.254:42335 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
I20260807 22:50:18.759027 13878 ts_manager.cc:194] Registered new tserver with Master: 6d336a0257c3479c9bd2ce5ad5c39412 (127.9.194.194:33827)
I20260807 22:50:18.760524 13878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:57573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:18.786788 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:19.125013 14232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:19.125401 14232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:19.125646 14232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:19.134716 14232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:19.134832 14232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:19.135044 14232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:19.144892 14232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:42335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:19.146194 14232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:19.147550 14232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:19.160835 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:19.160923 14237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:19.161131 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:19.161671 14232 server_base.cc:1034] running on GCE node
I20260807 22:50:19.162276 14232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:19.163291 14232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:19.164521 14232 hybrid_clock.cc:648] HybridClock initialized: now 1786143019164535 us; error 123 us; skew 500 ppm
I20260807 22:50:19.164899 14232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:19.168119 14232 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:19.169489 14232 webserver.cc:460] Webserver started at http://127.9.194.195:41951/ using document root <none> and password file <none>
I20260807 22:50:19.170158 14232 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:19.170297 14232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:19.170603 14232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:19.173650 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "a2502d73f4de4668a48ae6832fe903b8"
format_stamp: "Formatted at 2026-08-07 22:50:19 on dist-test-slave-5k54"
server_key: "c63b9444871dd2684fa703751714655c"
server_key_iv: "4093bdf7d344e12b453078d3e2b2e561"
server_key_version: "encryptionkey@0"
I20260807 22:50:19.174423 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "a2502d73f4de4668a48ae6832fe903b8"
format_stamp: "Formatted at 2026-08-07 22:50:19 on dist-test-slave-5k54"
server_key: "c63b9444871dd2684fa703751714655c"
server_key_iv: "4093bdf7d344e12b453078d3e2b2e561"
server_key_version: "encryptionkey@0"
I20260807 22:50:19.179687 14232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 22:50:19.183286 14248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:19.184657 14232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:19.184836 14232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "a2502d73f4de4668a48ae6832fe903b8"
format_stamp: "Formatted at 2026-08-07 22:50:19 on dist-test-slave-5k54"
server_key: "c63b9444871dd2684fa703751714655c"
server_key_iv: "4093bdf7d344e12b453078d3e2b2e561"
server_key_version: "encryptionkey@0"
I20260807 22:50:19.185057 14232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:19.211886 14232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:19.215969 14232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:19.216259 14232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:19.217262 14232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:19.218590 14232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:19.218691 14232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:19.218830 14232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:19.218900 14232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:19.253051 14232 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:45097
I20260807 22:50:19.253182 14371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:45097 every 8 connection(s)
I20260807 22:50:19.254350 14232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:19.258134 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14232
I20260807 22:50:19.258394 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:19.259037 9995 external_mini_cluster.cc:1428] Setting key ec11be6ead37f842658d295f3d3e4f76
I20260807 22:50:19.264513 14080 heartbeater.cc:503] Master 127.9.194.254:42335 was elected leader, sending a full tablet report...
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:19.275473 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.257151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:52735 (local address 127.9.194.254:42335)
0807 22:50:19.257620 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.257625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.258944 (+ 1319us) server_negotiation.cc:407] Connection header received
0807 22:50:19.261259 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.261264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.261359 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.261489 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.263030 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.263958 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.265728 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.266150 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.266280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.269614 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.269619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.269622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.269654 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.272362 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.273177 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.273182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.273185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.273311 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.273724 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.273732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.273736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.274007 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.274160 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.274830 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.275051 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":221,"threads_started":1}
I20260807 22:50:19.276476 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.257872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:42335 (local address 127.9.194.195:52735)
0807 22:50:19.258778 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:19.258816 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:19.261002 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:19.261741 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:19.261750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:19.262268 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:19.262835 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:19.262852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.264101 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.264106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:19.265589 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:19.265606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.266370 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.266378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:19.266624 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.267226 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:19.267248 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:19.269402 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:19.272500 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:19.272509 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:19.272525 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:19.273050 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:19.273420 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:19.273427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:19.273431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:19.273615 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:19.274133 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:19.274140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:19.274468 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 22:50:19.274742 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 22:50:19.275274 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":372,"threads_started":1}
I20260807 22:50:19.277757 14372 heartbeater.cc:347] Connected to a master server at 127.9.194.254:42335
I20260807 22:50:19.278067 14372 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:19.278770 14372 heartbeater.cc:511] Master 127.9.194.254:42335 requested a full tablet report, sending...
I20260807 22:50:19.280244 13878 ts_manager.cc:194] Registered new tserver with Master: a2502d73f4de4668a48ae6832fe903b8 (127.9.194.195:45097)
I20260807 22:50:19.281054 13878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:52735
I20260807 22:50:19.286633 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:19.340014 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.325204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.9.194.193:36315)
0807 22:50:19.325882 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.325896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.325925 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:50:19.325986 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.325993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.326210 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.326358 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.327832 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.328891 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.330088 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.330652 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.330749 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.334682 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.334694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.334709 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.334748 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.337545 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.338166 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.338173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.338175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.338296 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.338592 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.338595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.338597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.338921 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.339150 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.339347 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.339598 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":354,"threads_started":1}
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:19.358213 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.344468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.9.194.254:42335)
0807 22:50:19.344685 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.344691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.344877 (+ 186us) server_negotiation.cc:407] Connection header received
0807 22:50:19.345073 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.345077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.345157 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.345261 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.346484 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.347216 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.348511 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.348946 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.349082 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.352857 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.352862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.352865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.352922 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.355617 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.356246 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.356253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.356256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.356384 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.356751 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.356757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.356761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.357095 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.357356 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.357667 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.357926 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 22:50:19.358785 13878 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788
I20260807 22:50:19.359834 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13947
I20260807 22:50:19.372515 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.360741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.9.194.193:36315)
0807 22:50:19.360974 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.360980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.361223 (+ 243us) server_negotiation.cc:407] Connection header received
0807 22:50:19.361412 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.361417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.361490 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.361587 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.362855 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.363674 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.364857 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.365324 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.365452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.367614 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.367620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.367625 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.367664 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.369969 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.370598 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.370605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.370609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.370742 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.371107 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.371112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.371115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.371404 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.371667 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.371970 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.372197 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:50:19.373068 14074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:19.455207 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13947
I20260807 22:50:19.473541 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14087
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:19.490517 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.474510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53714 (local address 127.9.194.194:33827)
0807 22:50:19.475081 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.475088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.475113 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:50:19.475178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.475185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.475411 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.475638 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.476972 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.478056 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.479457 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.480027 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.480174 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.484447 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.484459 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.484474 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.484508 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.487626 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.488275 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.488283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.488285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.488418 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.488881 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.488886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.488889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.489236 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.489568 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.489884 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.490160 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":295,"threads_started":1}
I20260807 22:50:19.491881 14222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:19.574563 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14087
I20260807 22:50:19.592473 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14232
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:19.609338 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:19.593592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.9.194.195:45097)
0807 22:50:19.594242 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:19.594249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:19.594277 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:50:19.594335 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:19.594342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:19.594556 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:19.594721 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:19.596105 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.597109 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.598361 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:19.598938 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:19.599083 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:19.603663 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:19.603674 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:19.603687 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:19.603720 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:19.606671 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.607327 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.607334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.607336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.607520 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:19.607845 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:19.607849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:19.607851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:19.608182 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:19.608413 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:19.608729 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 22:50:19.608957 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":363,"threads_started":1}
I20260807 22:50:19.610605 14367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:19.694096 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14232
I20260807 22:50:19.711248 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 13845
I20260807 22:50:19.712496 13912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:19.817282 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 13845
2026-08-07T22:50:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4035 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[14395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[14395](info): set up 2 sockets
Aug 07 22:50:19 dist-test-slave-5k54 krb5kdc[14395](info): commencing operation
krb5kdc: starting...
W20260807 22:50:21.013036 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:21 dist-test-slave-5k54 krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:21.234601 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44159
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:44159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:21.558698 14408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:21.559006 14408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:21.559186 14408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:21.566390 14408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:21.566490 14408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:21.566587 14408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:21.566675 14408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:21.566756 14408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:21.576635 14408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:44159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:21.577905 14408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:21.579200 14408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:21.590623 14414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:21.590668 14413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:21.590714 14416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:21.591624 14408 server_base.cc:1034] running on GCE node
I20260807 22:50:21.592473 14408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:21.593802 14408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:21.595067 14408 hybrid_clock.cc:648] HybridClock initialized: now 1786143021595028 us; error 49 us; skew 500 ppm
I20260807 22:50:21.595402 14408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:21 dist-test-slave-5k54 krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:21.598500 14408 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:21.599740 14408 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:21.599906 14408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:21.600229 14408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:21.603690 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "7a39bc19562e44159f7ecdb4670bd638"
format_stamp: "Formatted at 2026-08-07 22:50:21 on dist-test-slave-5k54"
server_key: "2d6dc7461015658df8329bfd32c69264"
server_key_iv: "674f24a6f19c7cda4c727687145c031d"
server_key_version: "encryptionkey@0"
I20260807 22:50:21.604458 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "7a39bc19562e44159f7ecdb4670bd638"
format_stamp: "Formatted at 2026-08-07 22:50:21 on dist-test-slave-5k54"
server_key: "2d6dc7461015658df8329bfd32c69264"
server_key_iv: "674f24a6f19c7cda4c727687145c031d"
server_key_version: "encryptionkey@0"
I20260807 22:50:21.609447 14408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:50:21.612807 14422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:21.614131 14408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:50:21.614310 14408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "7a39bc19562e44159f7ecdb4670bd638"
format_stamp: "Formatted at 2026-08-07 22:50:21 on dist-test-slave-5k54"
server_key: "2d6dc7461015658df8329bfd32c69264"
server_key_iv: "674f24a6f19c7cda4c727687145c031d"
server_key_version: "encryptionkey@0"
I20260807 22:50:21.614523 14408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:21.627460 14408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:21.631546 14408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:21.631822 14408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:21.650306 14408 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:44159
I20260807 22:50:21.650329 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:44159 every 8 connection(s)
I20260807 22:50:21.651201 14408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:21.654425 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14408
I20260807 22:50:21.654675 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:21.654704 14476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:21.655306 9995 external_mini_cluster.cc:1428] Setting key 0747ed6c3a3f4fa7d218b1d718ecb84e
I20260807 22:50:21.669323 14476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: Bootstrap starting.
Aug 07 22:50:21 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:21.673310 14476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:21.674633 14476 log.cc:824] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:21.677878 14476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: No bootstrap required, opened a new log
I20260807 22:50:21.679975 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:21.657335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.9.194.254:44159)
0807 22:50:21.658291 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:21.658306 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:21.658347 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:21.660724 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:21.660745 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:21.661137 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:21.661465 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:21.663520 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:21.664851 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:21.666896 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:21.667609 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:21.667753 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:21.672682 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:21.672690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:21.672704 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:21.672746 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:21.675906 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:21.676681 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:21.676688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:21.676695 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:21.676861 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:21.677257 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:21.677263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:21.677266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:21.677787 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:21.678099 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:21.678566 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 22:50:21.678952 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":688,"thread_start_us":412,"threads_started":1}
I20260807 22:50:21.687343 14476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } }
I20260807 22:50:21.687816 14476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:21.687934 14476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a39bc19562e44159f7ecdb4670bd638, State: Initialized, Role: FOLLOWER
I20260807 22:50:21.688525 14476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } }
I20260807 22:50:21.688827 14476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:21.688971 14476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:21.689142 14476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:21.691962 14476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } }
I20260807 22:50:21.692498 14476 leader_election.cc:302] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a39bc19562e44159f7ecdb4670bd638; no voters:
I20260807 22:50:21.693101 14476 leader_election.cc:288] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:21.693197 14482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:21.694008 14482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [term 1 LEADER]: Becoming Leader. State: Replica: 7a39bc19562e44159f7ecdb4670bd638, State: Running, Role: LEADER
I20260807 22:50:21.694548 14482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } }
I20260807 22:50:21.695480 14476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:21.697412 14484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a39bc19562e44159f7ecdb4670bd638. Latest consensus state: current_term: 1 leader_uuid: "7a39bc19562e44159f7ecdb4670bd638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } } }
I20260807 22:50:21.697762 14484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:21.699697 14483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a39bc19562e44159f7ecdb4670bd638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39bc19562e44159f7ecdb4670bd638" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44159 } } }
I20260807 22:50:21.700069 14483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:21.700436 14490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:21.706650 14490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:21.715446 14490 catalog_manager.cc:1347] Generated new cluster ID: 61feb639fef8448791c82930a24392b5
I20260807 22:50:21.715561 14490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:21.728235 14490 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:21.730018 14490 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:21.736949 14490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: Generated new TSK 0
I20260807 22:50:21.737919 14490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:21.798712 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:22.137838 14505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:22.138123 14505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:22.138306 14505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:22.147637 14505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:22.147754 14505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:22.147958 14505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:22.157964 14505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:22.159397 14505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:22.160741 14505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:22.173573 14512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:22.173544 14510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:22.173527 14514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:22.174374 14505 server_base.cc:1034] running on GCE node
I20260807 22:50:22.174937 14505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:22.175902 14505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:22.177109 14505 hybrid_clock.cc:648] HybridClock initialized: now 1786143022177072 us; error 42 us; skew 500 ppm
I20260807 22:50:22.177525 14505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:22 dist-test-slave-5k54 krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143022, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:22.180747 14505 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:22.182017 14505 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:22.182156 14505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:22.182490 14505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:22.185488 14505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "1318bbdac2de4a879698d67443446612"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "f0449fe08b9f24be4c26b37dc12829a9"
server_key_iv: "29aa05de1c9d1cd9baedf7f850339356"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.186297 14505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "1318bbdac2de4a879698d67443446612"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "f0449fe08b9f24be4c26b37dc12829a9"
server_key_iv: "29aa05de1c9d1cd9baedf7f850339356"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.191215 14505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260807 22:50:22.194746 14520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.196087 14505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:22.196262 14505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "1318bbdac2de4a879698d67443446612"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "f0449fe08b9f24be4c26b37dc12829a9"
server_key_iv: "29aa05de1c9d1cd9baedf7f850339356"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.196537 14505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:22.213930 14505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:22.218122 14505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:22.218400 14505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:22.219463 14505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:22.220803 14505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:22.220932 14505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.221071 14505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:22.221161 14505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.254786 14505 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:40825
I20260807 22:50:22.254882 14638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:40825 every 8 connection(s)
I20260807 22:50:22.255776 14505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:22.261121 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14505
I20260807 22:50:22.261360 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:22.262001 9995 external_mini_cluster.cc:1428] Setting key da6eb5caa1b50e94660c9957eb020383
Aug 07 22:50:22 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143022, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:22.275519 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:22.258277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:40243 (local address 127.9.194.254:44159)
0807 22:50:22.258741 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:22.258746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:22.259659 (+ 913us) server_negotiation.cc:407] Connection header received
0807 22:50:22.261569 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:22.261576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:22.261678 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:22.261798 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:22.263317 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.264138 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.265685 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.266083 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.266213 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:22.269857 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:22.269864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:22.269869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:22.269911 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:22.272505 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:22.273214 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:22.273226 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:22.273229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:22.273350 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:22.273760 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:22.273766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:22.273770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:22.274032 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:22.274190 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:22.274881 (+ 691us) server_negotiation.cc:299] Negotiation successful
0807 22:50:22.275103 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":246,"threads_started":1}
I20260807 22:50:22.276445 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:22.258697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44159 (local address 127.9.194.193:40243)
0807 22:50:22.259503 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:22.259542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:22.261283 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:22.262023 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:22.262031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:22.262591 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:22.263143 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:22.263162 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.264288 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.264294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:22.265536 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:22.265548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.266314 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.266331 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:22.266553 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:22.267231 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:22.267253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:22.269651 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:22.272643 (+ 2992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:22.272653 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:22.272669 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:22.273091 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:22.273459 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:22.273464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:22.273468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:22.273647 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:22.274164 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:22.274172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:22.274492 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 22:50:22.274768 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 22:50:22.275310 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":303,"threads_started":1}
I20260807 22:50:22.278138 14639 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44159
I20260807 22:50:22.278486 14639 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:22.279310 14639 heartbeater.cc:511] Master 127.9.194.254:44159 requested a full tablet report, sending...
I20260807 22:50:22.281572 14440 ts_manager.cc:194] Registered new tserver with Master: 1318bbdac2de4a879698d67443446612 (127.9.194.193:40825)
I20260807 22:50:22.282979 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:40243
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:22.310633 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:22.643594 14646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:22.643965 14646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:22.644135 14646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:22.652547 14646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:22.652675 14646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:22.652860 14646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:22.668081 14646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:22.669430 14646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:22.670771 14646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:22.684376 14652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:22.684371 14653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:22.684279 14655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:22.685266 14646 server_base.cc:1034] running on GCE node
I20260807 22:50:22.685848 14646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:22.686748 14646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:22.687945 14646 hybrid_clock.cc:648] HybridClock initialized: now 1786143022687902 us; error 32 us; skew 500 ppm
I20260807 22:50:22.688273 14646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:22 dist-test-slave-5k54 krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143022, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:22.696120 14646 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:22.697432 14646 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:22.697569 14646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:22.697896 14646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:22.701067 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "14d32aa31955427198fec051ef23ba53"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "d2525e3ec630c40eaf6c28a2a8382e4f"
server_key_iv: "6f322eb045d39f57b474fb13997a827f"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.701932 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "14d32aa31955427198fec051ef23ba53"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "d2525e3ec630c40eaf6c28a2a8382e4f"
server_key_iv: "6f322eb045d39f57b474fb13997a827f"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.707039 14646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:50:22.710865 14661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.712253 14646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:22.712440 14646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "14d32aa31955427198fec051ef23ba53"
format_stamp: "Formatted at 2026-08-07 22:50:22 on dist-test-slave-5k54"
server_key: "d2525e3ec630c40eaf6c28a2a8382e4f"
server_key_iv: "6f322eb045d39f57b474fb13997a827f"
server_key_version: "encryptionkey@0"
I20260807 22:50:22.712666 14646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:22.734041 14646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:22.738397 14646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:22.738688 14646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:22.739768 14646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:22.741127 14646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:22.741215 14646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.741360 14646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:22.741456 14646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:22.774876 14646 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:34745
I20260807 22:50:22.774974 14781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:34745 every 8 connection(s)
I20260807 22:50:22.775851 14646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:22.780794 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14646
I20260807 22:50:22.781026 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:22.781606 9995 external_mini_cluster.cc:1428] Setting key f8787414ec1aee248546028882120465
Aug 07 22:50:22 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143022, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:22.799495 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:22.778526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:60573 (local address 127.9.194.254:44159)
0807 22:50:22.779088 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:22.779093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:22.780026 (+ 933us) server_negotiation.cc:407] Connection header received
0807 22:50:22.782541 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:22.782547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:22.782634 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:22.782762 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:22.784648 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.785373 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.787869 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.788287 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.788383 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:22.792978 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:22.792987 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:22.792994 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:22.793047 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:22.795994 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:22.796993 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:22.796997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:22.797000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:22.797139 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:22.797637 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:22.797640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:22.797642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:22.797935 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:22.798049 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:22.798994 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 22:50:22.799167 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":261,"threads_started":1}
I20260807 22:50:22.801138 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:22.779133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44159 (local address 127.9.194.194:60573)
0807 22:50:22.779869 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:22.779906 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:22.782244 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:22.782994 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:22.783003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:22.783648 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:22.784423 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:22.784454 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.785541 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.785550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:22.787728 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:22.787746 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:22.788519 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:22.788528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:22.788847 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:22.789598 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:22.789631 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:22.792628 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:22.796145 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:22.796156 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:22.796177 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:22.796880 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:22.797246 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:22.797253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:22.797259 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:22.797523 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:22.798073 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:22.798084 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:22.798559 (+ 475us) client_negotiation.cc:769] Sending connection context
0807 22:50:22.798866 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 22:50:22.799541 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":339,"threads_started":1}
I20260807 22:50:22.802670 14782 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44159
I20260807 22:50:22.803036 14782 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:22.803948 14782 heartbeater.cc:511] Master 127.9.194.254:44159 requested a full tablet report, sending...
I20260807 22:50:22.805771 14440 ts_manager.cc:194] Registered new tserver with Master: 14d32aa31955427198fec051ef23ba53 (127.9.194.194:34745)
I20260807 22:50:22.806587 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:60573
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:22.836614 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:23.164412 14790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:23.164678 14790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:23.164774 14790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:23.171976 14790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:23.172072 14790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:23.172185 14790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:23.181622 14790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:23.182824 14790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:23.184147 14790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:23.196858 14799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:23.197018 14796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:23.197018 14797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:23.197793 14790 server_base.cc:1034] running on GCE node
I20260807 22:50:23.198360 14790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:23.199290 14790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:23.200482 14790 hybrid_clock.cc:648] HybridClock initialized: now 1786143023200448 us; error 48 us; skew 500 ppm
I20260807 22:50:23.200785 14790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:23 dist-test-slave-5k54 krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143023, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:23.203986 14790 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:23.205158 14790 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:23.205269 14790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:23.205597 14790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:23.208554 14790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "20ca276612544b65afa03ca3b1308592"
format_stamp: "Formatted at 2026-08-07 22:50:23 on dist-test-slave-5k54"
server_key: "9a86a03bb03ef27e22952bb1de5e18d6"
server_key_iv: "20e468daa6da87041ebd76675865be19"
server_key_version: "encryptionkey@0"
I20260807 22:50:23.209363 14790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "20ca276612544b65afa03ca3b1308592"
format_stamp: "Formatted at 2026-08-07 22:50:23 on dist-test-slave-5k54"
server_key: "9a86a03bb03ef27e22952bb1de5e18d6"
server_key_iv: "20e468daa6da87041ebd76675865be19"
server_key_version: "encryptionkey@0"
I20260807 22:50:23.214383 14790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 22:50:23.219027 14805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:23.220896 14790 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:50:23.221083 14790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "20ca276612544b65afa03ca3b1308592"
format_stamp: "Formatted at 2026-08-07 22:50:23 on dist-test-slave-5k54"
server_key: "9a86a03bb03ef27e22952bb1de5e18d6"
server_key_iv: "20e468daa6da87041ebd76675865be19"
server_key_version: "encryptionkey@0"
I20260807 22:50:23.221354 14790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:23.239487 14790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:23.243496 14790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:23.243799 14790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:23.244760 14790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:23.246130 14790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:23.246254 14790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:23.246392 14790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:23.246486 14790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:23.281589 14790 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42607
I20260807 22:50:23.281646 14924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42607 every 8 connection(s)
I20260807 22:50:23.282474 14790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:23.286002 14639 heartbeater.cc:503] Master 127.9.194.254:44159 was elected leader, sending a full tablet report...
I20260807 22:50:23.288875 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14790
I20260807 22:50:23.289116 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:23.289552 9995 external_mini_cluster.cc:1428] Setting key b0ac8a119a14d85408bf019bf47432fc
Aug 07 22:50:23 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143023, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:23.303448 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.285030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:35749 (local address 127.9.194.254:44159)
0807 22:50:23.285207 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.285212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.286490 (+ 1278us) server_negotiation.cc:407] Connection header received
0807 22:50:23.288512 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.288518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.288618 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.288730 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:23.290386 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.291465 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.293105 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.293509 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.293605 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.296974 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:23.296994 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:23.297017 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:23.297078 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:23.299894 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.300779 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.300785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.300806 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.300946 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.301377 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.301382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.301386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.301807 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:23.302022 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.302752 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.303090 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:50:23.304296 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.285461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44159 (local address 127.9.194.195:35749)
0807 22:50:23.286290 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.286347 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.288222 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.288957 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.288965 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.289576 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:23.290170 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.290206 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.291620 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.291624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.292978 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.292990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.293697 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.293702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.293882 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.294515 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:23.294537 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:23.296781 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:23.300016 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:23.300023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:23.300035 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:23.300653 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:23.301053 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:23.301056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:23.301058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:23.301230 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:23.301998 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:23.302003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:23.302330 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.302597 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.302989 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":323,"threads_started":1}
I20260807 22:50:23.305760 14925 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44159
I20260807 22:50:23.306080 14925 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:23.306725 14925 heartbeater.cc:511] Master 127.9.194.254:44159 requested a full tablet report, sending...
I20260807 22:50:23.308132 14440 ts_manager.cc:194] Registered new tserver with Master: 20ca276612544b65afa03ca3b1308592 (127.9.194.195:42607)
I20260807 22:50:23.308923 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:35749
I20260807 22:50:23.317318 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:50:23.334667 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.322063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.9.194.254:44159)
0807 22:50:23.322289 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.322298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.322745 (+ 447us) server_negotiation.cc:407] Connection header received
0807 22:50:23.322874 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.322878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.322947 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.323029 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:23.324496 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.325239 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.326560 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.327071 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.327161 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.329397 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:23.329402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:23.329405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:23.329447 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:23.332116 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.332712 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.332716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.332719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.332839 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.333284 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.333287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.333288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.333604 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:23.333801 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.334109 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.334269 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:23.338915 14440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:23.345789 14440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:23.383824 14942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.358656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42607 (local address 127.0.0.1:59154)
0807 22:50:23.373456 (+ 14800us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.373488 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.374490 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.375603 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.375612 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.375642 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:23.376077 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.376094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.378036 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.378041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.380794 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.380809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.382751 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.382757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.382946 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.383005 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.383128 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.383331 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14607,"mutex_wait_us":70,"thread_start_us":16142,"threads_started":1}
I20260807 22:50:23.384008 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.359090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.9.194.195:42607)
0807 22:50:23.369226 (+ 10136us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.369240 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.373636 (+ 4396us) server_negotiation.cc:407] Connection header received
0807 22:50:23.374703 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.374715 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.375020 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.375231 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:23.376339 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.377895 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.380965 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.382214 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.382347 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.382630 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.383442 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.383621 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9893,"thread_start_us":9704,"threads_started":1}
I20260807 22:50:23.384513 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.358211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:34745 (local address 127.0.0.1:49564)
0807 22:50:23.373416 (+ 15205us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.373474 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.373671 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.377260 (+ 3589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.377264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.377291 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:23.377591 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.377602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.379296 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.379300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.381355 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.381370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.383229 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.383235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.383386 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.383413 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.383536 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.383665 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14871,"mutex_wait_us":67,"thread_start_us":15613,"threads_started":1}
I20260807 22:50:23.384745 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.358450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49564 (local address 127.9.194.194:34745)
0807 22:50:23.376681 (+ 18231us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.376691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.376720 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:50:23.376815 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.376822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.377054 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.377237 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:23.377753 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.379124 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.381522 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.382930 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.383071 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.383265 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.384041 (+ 776us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.384290 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18039,"thread_start_us":17883,"threads_started":1}
I20260807 22:50:23.385461 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.358951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40825 (local address 127.0.0.1:48856)
0807 22:50:23.375074 (+ 16123us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.375097 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.375220 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.376410 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.376414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.376440 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:23.376808 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.376816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.378738 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.378743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.381246 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.381256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.383487 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.383495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.383683 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.383717 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.383822 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.383949 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15999,"mutex_wait_us":173,"thread_start_us":14127,"threads_started":1}
I20260807 22:50:23.385787 14944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.359601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.9.194.193:40825)
0807 22:50:23.374630 (+ 15029us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.374639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.375355 (+ 716us) server_negotiation.cc:407] Connection header received
0807 22:50:23.375489 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.375496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.375770 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.376015 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:23.376981 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.378564 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.381400 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.382938 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.383071 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.383295 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.384266 (+ 971us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.384458 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14843,"thread_start_us":14650,"threads_started":1}
I20260807 22:50:23.388317 14857 tablet_service.cc:1467] Processing CreateTablet for tablet b9dee93068ac452b803aa972e5250250 (DEFAULT_TABLE table=test-table [id=adb444d1cf6d4b96ac900231925fb306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:23.388434 14710 tablet_service.cc:1467] Processing CreateTablet for tablet b9dee93068ac452b803aa972e5250250 (DEFAULT_TABLE table=test-table [id=adb444d1cf6d4b96ac900231925fb306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:23.389272 14571 tablet_service.cc:1467] Processing CreateTablet for tablet b9dee93068ac452b803aa972e5250250 (DEFAULT_TABLE table=test-table [id=adb444d1cf6d4b96ac900231925fb306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:23.389842 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9dee93068ac452b803aa972e5250250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:23.390677 14571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9dee93068ac452b803aa972e5250250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:23.390780 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9dee93068ac452b803aa972e5250250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:23.402309 14945 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Bootstrap starting.
I20260807 22:50:23.405830 14945 tablet_bootstrap.cc:654] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:23.406858 14945 log.cc:824] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:23.409499 14945 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: No bootstrap required, opened a new log
I20260807 22:50:23.409803 14945 ts_tablet_manager.cc:1397] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 22:50:23.410459 14946 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Bootstrap starting.
I20260807 22:50:23.411626 14947 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Bootstrap starting.
I20260807 22:50:23.414186 14946 tablet_bootstrap.cc:654] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:23.414532 14947 tablet_bootstrap.cc:654] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:23.415444 14946 log.cc:824] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:23.415858 14947 log.cc:824] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:23.418983 14946 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: No bootstrap required, opened a new log
I20260807 22:50:23.418984 14947 tablet_bootstrap.cc:492] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: No bootstrap required, opened a new log
I20260807 22:50:23.419251 14947 ts_tablet_manager.cc:1397] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 22:50:23.419279 14946 ts_tablet_manager.cc:1397] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 22:50:23.420887 14945 raft_consensus.cc:348] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.421343 14945 raft_consensus.cc:374] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:23.421450 14945 raft_consensus.cc:729] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca276612544b65afa03ca3b1308592, State: Initialized, Role: FOLLOWER
I20260807 22:50:23.422106 14945 consensus_queue.cc:260] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.423338 14925 heartbeater.cc:503] Master 127.9.194.254:44159 was elected leader, sending a full tablet report...
I20260807 22:50:23.424024 14945 ts_tablet_manager.cc:1428] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260807 22:50:23.428325 14946 raft_consensus.cc:348] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.428704 14946 raft_consensus.cc:374] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:23.428826 14946 raft_consensus.cc:729] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d32aa31955427198fec051ef23ba53, State: Initialized, Role: FOLLOWER
I20260807 22:50:23.429476 14946 consensus_queue.cc:260] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.429648 14947 raft_consensus.cc:348] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.430110 14947 raft_consensus.cc:374] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:23.430233 14947 raft_consensus.cc:729] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1318bbdac2de4a879698d67443446612, State: Initialized, Role: FOLLOWER
I20260807 22:50:23.430773 14782 heartbeater.cc:503] Master 127.9.194.254:44159 was elected leader, sending a full tablet report...
I20260807 22:50:23.431239 14946 ts_tablet_manager.cc:1428] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 22:50:23.431779 14947 consensus_queue.cc:260] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.434324 14947 ts_tablet_manager.cc:1428] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 22:50:23.449770 14952 raft_consensus.cc:482] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:23.450098 14952 raft_consensus.cc:504] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.451815 14952 leader_election.cc:288] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20ca276612544b65afa03ca3b1308592 (127.9.194.195:42607), 1318bbdac2de4a879698d67443446612 (127.9.194.193:40825)
I20260807 22:50:23.459540 14944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.452380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:46231 (local address 127.9.194.193:40825)
0807 22:50:23.452580 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.452587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.453269 (+ 682us) server_negotiation.cc:407] Connection header received
0807 22:50:23.453460 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.453465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.453544 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.453643 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:23.454533 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.455637 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.457826 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.458588 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.458782 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.458860 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.459129 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.459265 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:50:23.459736 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.452189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40825 (local address 127.9.194.194:46231)
0807 22:50:23.453116 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.453136 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.453310 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.453674 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.453679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.453717 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:23.454398 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.454414 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.455786 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.455794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.457667 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.457682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.458805 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.458811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.458961 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.458984 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.459105 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.459239 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":70,"spinlock_wait_cycles":3456,"thread_start_us":565,"threads_started":1}
I20260807 22:50:23.460027 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.452245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42607 (local address 127.9.194.194:59219)
0807 22:50:23.453185 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:23.453200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:23.453313 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:23.453674 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:23.453679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:23.453713 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:23.454281 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.454297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.455831 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.455835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.457917 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:23.457930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.459081 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.459087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:23.459257 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.459280 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:50:23.459395 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:50:23.459535 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":20,"spinlock_wait_cycles":30848,"thread_start_us":415,"threads_started":1}
I20260807 22:50:23.460278 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.452232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59219 (local address 127.9.194.195:42607)
0807 22:50:23.452468 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.452475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.453293 (+ 818us) server_negotiation.cc:407] Connection header received
0807 22:50:23.453460 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.453465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.453543 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.453643 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:23.454487 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.455644 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.458078 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.458866 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.459062 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.459137 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.459449 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.459581 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:23.460891 14592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9dee93068ac452b803aa972e5250250" candidate_uuid: "14d32aa31955427198fec051ef23ba53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1318bbdac2de4a879698d67443446612" is_pre_election: true
I20260807 22:50:23.461103 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9dee93068ac452b803aa972e5250250" candidate_uuid: "14d32aa31955427198fec051ef23ba53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca276612544b65afa03ca3b1308592" is_pre_election: true
I20260807 22:50:23.461421 14592 raft_consensus.cc:2454] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d32aa31955427198fec051ef23ba53 in term 0.
I20260807 22:50:23.461674 14877 raft_consensus.cc:2454] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d32aa31955427198fec051ef23ba53 in term 0.
I20260807 22:50:23.462150 14665 leader_election.cc:302] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1318bbdac2de4a879698d67443446612, 14d32aa31955427198fec051ef23ba53; no voters:
I20260807 22:50:23.462566 14952 raft_consensus.cc:2781] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:23.462688 14952 raft_consensus.cc:482] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:23.462762 14952 raft_consensus.cc:3037] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:23.464962 14952 raft_consensus.cc:504] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.465543 14952 leader_election.cc:288] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [CANDIDATE]: Term 1 election: Requested vote from peers 20ca276612544b65afa03ca3b1308592 (127.9.194.195:42607), 1318bbdac2de4a879698d67443446612 (127.9.194.193:40825)
I20260807 22:50:23.465893 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9dee93068ac452b803aa972e5250250" candidate_uuid: "14d32aa31955427198fec051ef23ba53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca276612544b65afa03ca3b1308592"
I20260807 22:50:23.466032 14592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9dee93068ac452b803aa972e5250250" candidate_uuid: "14d32aa31955427198fec051ef23ba53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1318bbdac2de4a879698d67443446612"
I20260807 22:50:23.466109 14877 raft_consensus.cc:3037] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:23.466240 14592 raft_consensus.cc:3037] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:23.469415 14592 raft_consensus.cc:2454] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d32aa31955427198fec051ef23ba53 in term 1.
I20260807 22:50:23.469424 14877 raft_consensus.cc:2454] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d32aa31955427198fec051ef23ba53 in term 1.
I20260807 22:50:23.469818 14665 leader_election.cc:302] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1318bbdac2de4a879698d67443446612, 14d32aa31955427198fec051ef23ba53, 20ca276612544b65afa03ca3b1308592; no voters:
I20260807 22:50:23.470135 14952 raft_consensus.cc:2781] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:23.470652 14952 raft_consensus.cc:686] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 1 LEADER]: Becoming Leader. State: Replica: 14d32aa31955427198fec051ef23ba53, State: Running, Role: LEADER
I20260807 22:50:23.471133 14952 consensus_queue.cc:237] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } }
I20260807 22:50:23.475903 14439 catalog_manager.cc:5495] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14d32aa31955427198fec051ef23ba53 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "14d32aa31955427198fec051ef23ba53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d32aa31955427198fec051ef23ba53" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 } health_report { overall_health: UNKNOWN } } }
W20260807 22:50:23.508070 14641 tablet.cc:2369] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:23.512631 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.507565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.9.194.194:34745)
0807 22:50:23.507792 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.507804 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.507821 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:50:23.507903 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.507907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.507978 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.508146 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:23.508515 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.509182 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.510531 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.510929 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.511058 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.511579 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:23.512055 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:23.512214 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.512289 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.512370 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:23.521035 14877 raft_consensus.cc:1261] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 1 FOLLOWER]: Refusing update from remote peer 14d32aa31955427198fec051ef23ba53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:23.521063 14592 raft_consensus.cc:1261] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 1 FOLLOWER]: Refusing update from remote peer 14d32aa31955427198fec051ef23ba53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:23.521982 14957 consensus_queue.cc:1035] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1318bbdac2de4a879698d67443446612" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:23.522326 14952 consensus_queue.cc:1035] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca276612544b65afa03ca3b1308592" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:50:23.528047 14784 tablet.cc:2369] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:23.535704 14961 mvcc.cc:205] Tried to move back new op lower bound from 7316041824331657216 to 7316041824142372864. Current Snapshot: MvccSnapshot[applied={T|T < 7316041824331657216}]
I20260807 22:50:23.536109 14960 mvcc.cc:205] Tried to move back new op lower bound from 7316041824331657216 to 7316041824142372864. Current Snapshot: MvccSnapshot[applied={T|T < 7316041824331657216 or (T in {7316041824331657216})}]
I20260807 22:50:23.548370 14440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:23.548694 14440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:23.551973 14440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a39bc19562e44159f7ecdb4670bd638: Sending DeleteTablet for 3 replicas of tablet b9dee93068ac452b803aa972e5250250
I20260807 22:50:23.553064 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet b9dee93068ac452b803aa972e5250250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:23 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:49564
I20260807 22:50:23.553136 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet b9dee93068ac452b803aa972e5250250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:23 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:59154
I20260807 22:50:23.553844 14571 tablet_service.cc:1514] Processing DeleteTablet for tablet b9dee93068ac452b803aa972e5250250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:23 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:48856
I20260807 22:50:23.554096 14969 tablet_replica.cc:330] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: stopping tablet replica
I20260807 22:50:23.554622 14969 raft_consensus.cc:2229] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:23.555414 14969 raft_consensus.cc:2258] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:23.555676 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14505
I20260807 22:50:23.557868 14969 ts_tablet_manager.cc:1905] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:23.567739 14969 ts_tablet_manager.cc:1918] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:23.568079 14969 log.cc:1196] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/b9dee93068ac452b803aa972e5250250
I20260807 22:50:23.568934 14969 ts_tablet_manager.cc:1939] T b9dee93068ac452b803aa972e5250250 P 14d32aa31955427198fec051ef23ba53: Deleting consensus metadata
I20260807 22:50:23.569563 14971 tablet_replica.cc:330] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: stopping tablet replica
I20260807 22:50:23.570050 14971 raft_consensus.cc:2229] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:23.570595 14971 raft_consensus.cc:2258] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:23.571154 14424 catalog_manager.cc:4841] TS 14d32aa31955427198fec051ef23ba53 (127.9.194.194:34745): tablet b9dee93068ac452b803aa972e5250250 (table test-table [id=adb444d1cf6d4b96ac900231925fb306]) successfully deleted
I20260807 22:50:23.573056 14971 ts_tablet_manager.cc:1905] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:23.573464 14970 tablet_replica.cc:330] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: stopping tablet replica
I20260807 22:50:23.573859 14970 raft_consensus.cc:2229] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:23.574355 14970 raft_consensus.cc:2258] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:23.579278 14971 ts_tablet_manager.cc:1918] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:23.579303 14970 ts_tablet_manager.cc:1905] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:23.579527 14971 log.cc:1196] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/b9dee93068ac452b803aa972e5250250
I20260807 22:50:23.580263 14971 ts_tablet_manager.cc:1939] T b9dee93068ac452b803aa972e5250250 P 1318bbdac2de4a879698d67443446612: Deleting consensus metadata
Aug 07 22:50:23 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:23.582278 14426 catalog_manager.cc:4841] TS 1318bbdac2de4a879698d67443446612 (127.9.194.193:40825): tablet b9dee93068ac452b803aa972e5250250 (table test-table [id=adb444d1cf6d4b96ac900231925fb306]) successfully deleted
I20260807 22:50:23.586633 14970 ts_tablet_manager.cc:1918] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:23.586826 14970 log.cc:1196] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/b9dee93068ac452b803aa972e5250250
I20260807 22:50:23.587399 14970 ts_tablet_manager.cc:1939] T b9dee93068ac452b803aa972e5250250 P 20ca276612544b65afa03ca3b1308592: Deleting consensus metadata
I20260807 22:50:23.589112 14944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.556800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.9.194.193:40825)
0807 22:50:23.556967 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.556974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.566551 (+ 9577us) server_negotiation.cc:407] Connection header received
0807 22:50:23.566744 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.566750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.566849 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.566962 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:23.572369 (+ 5407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.573464 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.576559 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.577256 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.577441 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.583099 (+ 5658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:23.583110 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:23.583124 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:23.583158 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:23.586051 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.586764 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.586782 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.586789 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.587001 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.587398 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.587403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.587408 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.587869 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:23.588207 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.588476 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.588712 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:50:23.589318 14423 catalog_manager.cc:4841] TS 20ca276612544b65afa03ca3b1308592 (127.9.194.195:42607): tablet b9dee93068ac452b803aa972e5250250 (table test-table [id=adb444d1cf6d4b96ac900231925fb306]) successfully deleted
I20260807 22:50:23.590096 14633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:23.681267 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14505
I20260807 22:50:23.701932 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14646
Aug 07 22:50:23 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:23.719652 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.702791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.9.194.194:34745)
0807 22:50:23.703094 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.703101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.703123 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:50:23.703197 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.703203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.703295 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.703415 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:23.704638 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.705873 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.707230 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.708021 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.708153 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.712239 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:23.712255 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:23.712280 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:23.712332 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:23.716314 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.717056 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.717116 (+ 60us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.717119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.717322 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.717900 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.717909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.717915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.718411 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:23.718759 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.718991 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.719231 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260807 22:50:23.720296 14777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:23.834964 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14646
I20260807 22:50:23.856127 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14790
Aug 07 22:50:23 dist-test-slave-5k54 krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:23.871308 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:23.856915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.9.194.195:42607)
0807 22:50:23.857097 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:23.857102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:23.857121 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:23.857247 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:23.857252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:23.857317 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:23.857400 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:23.858644 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.859367 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.860606 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:23.861000 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:23.861167 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:23.865728 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:23.865755 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:23.865772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:23.865808 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:23.868674 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.869474 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.869482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.869484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.869632 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:23.870017 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:23.870020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:23.870022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:23.870352 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:23.870591 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:23.870854 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 22:50:23.871062 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:50:23.872125 14920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:23.960997 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14790
I20260807 22:50:23.979456 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14408
I20260807 22:50:23.980633 14471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:24.090009 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14408
2026-08-07T22:50:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4273 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T22:50:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:24Z Disabled control of system clock
I20260807 22:50:24.164383 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43833
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:50:24.489326 14983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:24.489622 14983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:24.489806 14983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:24.497149 14983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:24.497251 14983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:24.497341 14983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:24.497431 14983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:24.497517 14983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:24.506762 14983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:24.508437 14983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:24.510334 14983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:24.522090 14991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:24.522115 14989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:24.522385 14988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:24.523221 14983 server_base.cc:1034] running on GCE node
I20260807 22:50:24.524144 14983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:24.525508 14983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:24.526718 14983 hybrid_clock.cc:648] HybridClock initialized: now 1786143024526674 us; error 30 us; skew 500 ppm
I20260807 22:50:24.527112 14983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:24.529502 14983 webserver.cc:460] Webserver started at http://127.9.194.254:45503/ using document root <none> and password file <none>
I20260807 22:50:24.530190 14983 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:24.530303 14983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:24.530635 14983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:24.533603 14983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "b26897c71a3b4ea5a4ff0b5df854fccd"
format_stamp: "Formatted at 2026-08-07 22:50:24 on dist-test-slave-5k54"
server_key: "f8f3acf9f6819aa28b7fe7076fe039fc"
server_key_iv: "f6ecb46bb5e723685dc5c7e66ab21010"
server_key_version: "encryptionkey@0"
I20260807 22:50:24.534426 14983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "b26897c71a3b4ea5a4ff0b5df854fccd"
format_stamp: "Formatted at 2026-08-07 22:50:24 on dist-test-slave-5k54"
server_key: "f8f3acf9f6819aa28b7fe7076fe039fc"
server_key_iv: "f6ecb46bb5e723685dc5c7e66ab21010"
server_key_version: "encryptionkey@0"
I20260807 22:50:24.539672 14983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260807 22:50:24.543057 14997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:24.544433 14983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:24.544617 14983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "b26897c71a3b4ea5a4ff0b5df854fccd"
format_stamp: "Formatted at 2026-08-07 22:50:24 on dist-test-slave-5k54"
server_key: "f8f3acf9f6819aa28b7fe7076fe039fc"
server_key_iv: "f6ecb46bb5e723685dc5c7e66ab21010"
server_key_version: "encryptionkey@0"
I20260807 22:50:24.544848 14983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:24.550374 14983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:24.551196 14983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:24.551551 14983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:24.569283 14983 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43833
I20260807 22:50:24.569315 15055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43833 every 8 connection(s)
I20260807 22:50:24.570689 14983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:24.572584 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 14983
I20260807 22:50:24.572794 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:24.573539 9995 external_mini_cluster.cc:1428] Setting key d2d986d3dcabb088a155cd2d45ca13d6
I20260807 22:50:24.574940 15056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:24.582703 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:24.575645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.9.194.254:43833)
0807 22:50:24.576779 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:24.576804 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:24.576856 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:50:24.578575 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:24.578602 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:24.579161 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:24.579621 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:24.580666 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:24.580674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:24.580691 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:24.580706 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:24.580815 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:24.580837 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:24.581068 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:24.581211 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 22:50:24.581482 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":364,"threads_started":1}
I20260807 22:50:24.589335 15056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: Bootstrap starting.
I20260807 22:50:24.592428 15056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:24.593549 15056 log.cc:824] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:24.596161 15056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: No bootstrap required, opened a new log
I20260807 22:50:24.603377 15056 raft_consensus.cc:348] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } }
I20260807 22:50:24.603789 15056 raft_consensus.cc:374] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:24.603866 15056 raft_consensus.cc:729] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26897c71a3b4ea5a4ff0b5df854fccd, State: Initialized, Role: FOLLOWER
I20260807 22:50:24.604405 15056 consensus_queue.cc:260] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } }
I20260807 22:50:24.604610 15056 raft_consensus.cc:388] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:24.604708 15056 raft_consensus.cc:482] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:24.604849 15056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:24.607162 15056 raft_consensus.cc:504] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } }
I20260807 22:50:24.607663 15056 leader_election.cc:302] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b26897c71a3b4ea5a4ff0b5df854fccd; no voters:
I20260807 22:50:24.608233 15056 leader_election.cc:288] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:24.608358 15062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:24.609059 15062 raft_consensus.cc:686] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [term 1 LEADER]: Becoming Leader. State: Replica: b26897c71a3b4ea5a4ff0b5df854fccd, State: Running, Role: LEADER
I20260807 22:50:24.609609 15062 consensus_queue.cc:237] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } }
I20260807 22:50:24.610316 15056 sys_catalog.cc:564] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:24.612593 15064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b26897c71a3b4ea5a4ff0b5df854fccd. Latest consensus state: current_term: 1 leader_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } } }
I20260807 22:50:24.613243 15064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:24.613276 15063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26897c71a3b4ea5a4ff0b5df854fccd" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43833 } } }
I20260807 22:50:24.613624 15063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:24.620291 15070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:24.625383 15070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:24.634373 15070 catalog_manager.cc:1347] Generated new cluster ID: 06602e9417f24d79880dc3eb709ef320
I20260807 22:50:24.634502 15070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:24.649330 15070 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:24.649848 15070 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:24.663674 15070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: Generated new TSK 0
I20260807 22:50:24.664708 15070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:50:24.680688 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--builtin_ntp_servers=127.9.194.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:50:25.018926 15082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:25.019253 15082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:25.019456 15082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:25.026701 15082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:25.026847 15082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:25.027050 15082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:25.036352 15082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:25.037685 15082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:25.039513 15082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:25.052582 15090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:25.052701 15087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:25.052690 15088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:25.053716 15082 server_base.cc:1034] running on GCE node
I20260807 22:50:25.054342 15082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:25.055279 15082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:25.056504 15082 hybrid_clock.cc:648] HybridClock initialized: now 1786143025056474 us; error 70 us; skew 500 ppm
I20260807 22:50:25.056818 15082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:25.059110 15082 webserver.cc:460] Webserver started at http://127.9.194.193:33775/ using document root <none> and password file <none>
I20260807 22:50:25.059878 15082 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:25.060026 15082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:25.060346 15082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:25.063259 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "2277985909fd445f8845d4b7b2963204"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "20eac3a5d4054bac3c52e08e7e2f12f3"
server_key_iv: "153c1c2ec4913b660e951c8402f2f40a"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.064167 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "2277985909fd445f8845d4b7b2963204"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "20eac3a5d4054bac3c52e08e7e2f12f3"
server_key_iv: "153c1c2ec4913b660e951c8402f2f40a"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.069237 15082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:25.072726 15096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.074239 15082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:50:25.074414 15082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "2277985909fd445f8845d4b7b2963204"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "20eac3a5d4054bac3c52e08e7e2f12f3"
server_key_iv: "153c1c2ec4913b660e951c8402f2f40a"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.074645 15082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:25.080515 15082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:25.081393 15082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:25.081668 15082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:25.082592 15082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:25.084182 15082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:25.084303 15082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.084438 15082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:25.084537 15082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.118108 15082 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:38621
I20260807 22:50:25.118170 15219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:38621 every 8 connection(s)
I20260807 22:50:25.119652 15082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:25.124830 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15082
I20260807 22:50:25.125066 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:25.125717 9995 external_mini_cluster.cc:1428] Setting key 0ac0e98ffe2f61861678caa4540538d9
I20260807 22:50:25.128345 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:25.122275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:51279 (local address 127.9.194.254:43833)
0807 22:50:25.122715 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:25.122719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:25.124031 (+ 1312us) server_negotiation.cc:407] Connection header received
0807 22:50:25.125897 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:25.125902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:25.125974 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:25.126107 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:25.127545 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:25.127551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:25.127556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:25.127574 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:25.127628 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:25.127640 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:25.127771 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:25.127989 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:50:25.128059 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":224,"threads_started":1}
I20260807 22:50:25.128666 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--builtin_ntp_servers=127.9.194.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 22:50:25.129377 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:25.122908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43833 (local address 127.9.194.193:51279)
0807 22:50:25.123861 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:25.123898 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:25.125622 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:25.126367 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:25.126376 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:25.126473 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:25.127225 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:25.127228 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:25.127264 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:25.127353 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:25.127735 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:25.127748 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:25.127757 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 22:50:25.128077 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 22:50:25.128340 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":277,"threads_started":1}
I20260807 22:50:25.130982 15220 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43833
I20260807 22:50:25.131237 15220 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:25.131852 15220 heartbeater.cc:511] Master 127.9.194.254:43833 requested a full tablet report, sending...
I20260807 22:50:25.134112 15015 ts_manager.cc:194] Registered new tserver with Master: 2277985909fd445f8845d4b7b2963204 (127.9.194.193:38621)
W20260807 22:50:25.456255 15224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:25.456597 15224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:25.456769 15224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:25.463980 15224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:25.464104 15224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:25.464282 15224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:25.473204 15224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:25.474486 15224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:25.476325 15224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:25.489635 15229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:25.489635 15230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:25.490128 15224 server_base.cc:1034] running on GCE node
W20260807 22:50:25.490319 15232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:25.491014 15224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:25.491940 15224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:25.493149 15224 hybrid_clock.cc:648] HybridClock initialized: now 1786143025493089 us; error 52 us; skew 500 ppm
I20260807 22:50:25.493492 15224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:25.495678 15224 webserver.cc:460] Webserver started at http://127.9.194.194:36515/ using document root <none> and password file <none>
I20260807 22:50:25.496356 15224 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:25.496469 15224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:25.496807 15224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:25.499720 15224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "ef38c40b5a424d7b95b405ba827c9d76"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "6f06362d546b059a401c28f0984f6b34"
server_key_iv: "493c2c57720aa1518d509f8a2701c6c1"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.500504 15224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "ef38c40b5a424d7b95b405ba827c9d76"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "6f06362d546b059a401c28f0984f6b34"
server_key_iv: "493c2c57720aa1518d509f8a2701c6c1"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.505702 15224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:25.509130 15238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.510421 15224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:50:25.510589 15224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "ef38c40b5a424d7b95b405ba827c9d76"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "6f06362d546b059a401c28f0984f6b34"
server_key_iv: "493c2c57720aa1518d509f8a2701c6c1"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.510860 15224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:25.516554 15224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:25.517326 15224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:25.517602 15224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:25.518617 15224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:25.520123 15224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:25.520246 15224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.520376 15224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:25.520462 15224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.553344 15224 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:37111
I20260807 22:50:25.553398 15360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:37111 every 8 connection(s)
I20260807 22:50:25.554694 15224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:25.561156 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15224
I20260807 22:50:25.561328 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:25.561944 9995 external_mini_cluster.cc:1428] Setting key 452c1c077e412fb06a3602dab265411e
I20260807 22:50:25.563094 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:25.557328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:43413 (local address 127.9.194.254:43833)
0807 22:50:25.557491 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:25.557496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:25.558891 (+ 1395us) server_negotiation.cc:407] Connection header received
0807 22:50:25.560700 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:25.560704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:25.560774 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:25.560856 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:25.562359 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:25.562363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:25.562366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:25.562381 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:25.562421 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:25.562433 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:25.562522 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:25.562753 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 22:50:25.562824 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:50:25.564203 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:25.557805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43833 (local address 127.9.194.194:43413)
0807 22:50:25.558722 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:25.558758 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:25.560448 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:25.561141 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:25.561153 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:25.561278 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:25.561990 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:25.561994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:25.562046 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:25.562159 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:25.562532 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:25.562544 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:25.562553 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 22:50:25.562781 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 22:50:25.563061 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":258,"threads_started":1}
I20260807 22:50:25.565083 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--builtin_ntp_servers=127.9.194.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 22:50:25.565543 15361 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43833
I20260807 22:50:25.565851 15361 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:25.566419 15361 heartbeater.cc:511] Master 127.9.194.254:43833 requested a full tablet report, sending...
I20260807 22:50:25.567917 15015 ts_manager.cc:194] Registered new tserver with Master: ef38c40b5a424d7b95b405ba827c9d76 (127.9.194.194:37111)
W20260807 22:50:25.906268 15365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:25.906544 15365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:25.906636 15365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:25.913968 15365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:25.914075 15365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:25.914183 15365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:25.923285 15365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:25.924539 15365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:25.926321 15365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:25.939455 15371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:25.939662 15372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:25.939704 15374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:25.940666 15365 server_base.cc:1034] running on GCE node
I20260807 22:50:25.941351 15365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:25.942382 15365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:25.943624 15365 hybrid_clock.cc:648] HybridClock initialized: now 1786143025943557 us; error 65 us; skew 500 ppm
I20260807 22:50:25.943966 15365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:25.946488 15365 webserver.cc:460] Webserver started at http://127.9.194.195:40109/ using document root <none> and password file <none>
I20260807 22:50:25.947216 15365 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:25.947355 15365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:25.947719 15365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:25.950773 15365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "2e0c3857b25d465bb079b79f071870b8"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "8948e01fe6e6ee5bd903bdc8e69ef595"
server_key_iv: "72ef517978e9f3e4eb32c917590c7790"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.951668 15365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "2e0c3857b25d465bb079b79f071870b8"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "8948e01fe6e6ee5bd903bdc8e69ef595"
server_key_iv: "72ef517978e9f3e4eb32c917590c7790"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.956976 15365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:50:25.960582 15380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.961982 15365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:50:25.962109 15365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "2e0c3857b25d465bb079b79f071870b8"
format_stamp: "Formatted at 2026-08-07 22:50:25 on dist-test-slave-5k54"
server_key: "8948e01fe6e6ee5bd903bdc8e69ef595"
server_key_iv: "72ef517978e9f3e4eb32c917590c7790"
server_key_version: "encryptionkey@0"
I20260807 22:50:25.962301 15365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:25.968070 15365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:25.968783 15365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:25.969022 15365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:25.969839 15365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:25.971131 15365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:25.971216 15365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:25.971299 15365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:25.971356 15365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:26.004657 15365 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:35993
I20260807 22:50:26.004732 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:35993 every 8 connection(s)
I20260807 22:50:26.006054 15365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:26.010563 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15365
I20260807 22:50:26.010746 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:26.011349 9995 external_mini_cluster.cc:1428] Setting key a362ca35ccccc471f32997e2ccb4dfbf
I20260807 22:50:26.015254 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.008723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:54769 (local address 127.9.194.254:43833)
0807 22:50:26.008910 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.008915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.010477 (+ 1562us) server_negotiation.cc:407] Connection header received
0807 22:50:26.012760 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.012766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.012836 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.012960 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.014482 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.014488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.014493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.014512 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.014556 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.014568 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.014688 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.014887 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.014967 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:50:26.016232 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.009237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43833 (local address 127.9.194.195:54769)
0807 22:50:26.010285 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.010336 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.012456 (+ 2120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.013203 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.013230 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.013329 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.014123 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.014126 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.014178 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.014295 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.014662 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.014673 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.014681 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.014997 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.015264 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":269,"threads_started":1}
I20260807 22:50:26.017463 15501 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43833
I20260807 22:50:26.017724 15501 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:26.018329 15501 heartbeater.cc:511] Master 127.9.194.254:43833 requested a full tablet report, sending...
I20260807 22:50:26.019765 15015 ts_manager.cc:194] Registered new tserver with Master: 2e0c3857b25d465bb079b79f071870b8 (127.9.194.195:35993)
I20260807 22:50:26.026800 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:50:26.033861 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.031469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.9.194.254:43833)
0807 22:50:26.031634 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.031639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.032182 (+ 543us) server_negotiation.cc:407] Connection header received
0807 22:50:26.032318 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.032322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.032372 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.032462 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.033330 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.033334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.033337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.033351 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.033381 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.033392 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.033502 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.033567 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.033611 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:50:26.037935 15015 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:26.044826 15015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:26.073469 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.057381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.9.194.195:35993)
0807 22:50:26.070023 (+ 12642us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.070035 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.070076 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:26.070151 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.070159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.070407 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.070577 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.072357 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.072368 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.072382 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.072400 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.072463 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.072484 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.072669 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.072909 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.073016 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12462,"thread_start_us":12297,"threads_started":1}
I20260807 22:50:26.073496 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.057181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:35993 (local address 127.0.0.1:51840)
0807 22:50:26.068061 (+ 10880us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.068116 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.068324 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.070587 (+ 2263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.070592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.070621 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.072100 (+ 1479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.072106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.072125 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.072169 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.072609 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.072621 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.072629 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.072833 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.072948 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9413,"thread_start_us":11344,"threads_started":1}
I20260807 22:50:26.073624 15516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.060340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42912)
0807 22:50:26.068827 (+ 8487us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.068850 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.068970 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.071447 (+ 2477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.071452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.071476 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.072237 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.072243 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.072263 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.072313 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.072957 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.072961 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.072966 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.073114 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.073185 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8336,"thread_start_us":6033,"threads_started":1}
I20260807 22:50:26.073859 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.057672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.9.194.194:37111)
0807 22:50:26.069586 (+ 11914us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.069601 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.069643 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:50:26.069733 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.069745 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.070059 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.071098 (+ 1039us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.072495 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.072503 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.072516 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.072535 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.072589 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.072611 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.072796 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.073411 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.073525 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11678,"thread_start_us":11504,"threads_started":1}
I20260807 22:50:26.074088 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.057540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:37111 (local address 127.0.0.1:59750)
0807 22:50:26.068528 (+ 10988us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.068550 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.068676 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.070282 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.070287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.070367 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.071468 (+ 1101us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.071472 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.071513 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.071602 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.072774 (+ 1172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.072779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.072783 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.072943 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.072999 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10808,"thread_start_us":10607,"threads_started":1}
I20260807 22:50:26.074549 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.060575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.9.194.193:38621)
0807 22:50:26.070274 (+ 9699us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.070281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.070305 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:50:26.070369 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.070375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.070557 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.070692 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.072657 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.072666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.072677 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.072693 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.072753 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.072780 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.072906 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.074012 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.074225 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9555,"thread_start_us":9439,"threads_started":1}
I20260807 22:50:26.078377 15149 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6c2f848904afd9426789642b1f008 (DEFAULT_TABLE table=test-table [id=d566231b81c449d4b005b011252670aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:26.078341 15431 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6c2f848904afd9426789642b1f008 (DEFAULT_TABLE table=test-table [id=d566231b81c449d4b005b011252670aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:26.078341 15291 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6c2f848904afd9426789642b1f008 (DEFAULT_TABLE table=test-table [id=d566231b81c449d4b005b011252670aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:26.079747 15149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6c2f848904afd9426789642b1f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:26.079782 15431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6c2f848904afd9426789642b1f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:26.079783 15291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6c2f848904afd9426789642b1f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:26.091811 15520 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Bootstrap starting.
I20260807 22:50:26.096215 15522 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Bootstrap starting.
I20260807 22:50:26.097064 15521 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Bootstrap starting.
I20260807 22:50:26.097150 15520 tablet_bootstrap.cc:654] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:26.098484 15520 log.cc:824] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:26.100884 15521 tablet_bootstrap.cc:654] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:26.101538 15520 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: No bootstrap required, opened a new log
I20260807 22:50:26.101818 15520 ts_tablet_manager.cc:1397] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 22:50:26.102192 15521 log.cc:824] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:26.102905 15522 tablet_bootstrap.cc:654] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:26.104825 15522 log.cc:824] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:26.105906 15521 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: No bootstrap required, opened a new log
I20260807 22:50:26.106184 15521 ts_tablet_manager.cc:1397] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:50:26.108109 15522 tablet_bootstrap.cc:492] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: No bootstrap required, opened a new log
I20260807 22:50:26.108385 15522 ts_tablet_manager.cc:1397] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 22:50:26.110289 15520 raft_consensus.cc:348] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.110733 15520 raft_consensus.cc:374] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:26.110857 15520 raft_consensus.cc:729] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2277985909fd445f8845d4b7b2963204, State: Initialized, Role: FOLLOWER
I20260807 22:50:26.111634 15520 consensus_queue.cc:260] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.112843 15220 heartbeater.cc:503] Master 127.9.194.254:43833 was elected leader, sending a full tablet report...
I20260807 22:50:26.113546 15520 ts_tablet_manager.cc:1428] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 22:50:26.117748 15521 raft_consensus.cc:348] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.118233 15521 raft_consensus.cc:374] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:26.118383 15521 raft_consensus.cc:729] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef38c40b5a424d7b95b405ba827c9d76, State: Initialized, Role: FOLLOWER
I20260807 22:50:26.119071 15521 consensus_queue.cc:260] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.119880 15522 raft_consensus.cc:348] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.120370 15522 raft_consensus.cc:374] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:26.120718 15361 heartbeater.cc:503] Master 127.9.194.254:43833 was elected leader, sending a full tablet report...
I20260807 22:50:26.121371 15521 ts_tablet_manager.cc:1428] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 22:50:26.120479 15522 raft_consensus.cc:729] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0c3857b25d465bb079b79f071870b8, State: Initialized, Role: FOLLOWER
W20260807 22:50:26.121848 15221 tablet.cc:2369] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:26.123538 15522 consensus_queue.cc:260] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.124693 15501 heartbeater.cc:503] Master 127.9.194.254:43833 was elected leader, sending a full tablet report...
I20260807 22:50:26.126296 15522 ts_tablet_manager.cc:1428] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
W20260807 22:50:26.257877 15502 tablet.cc:2369] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:26.302204 15526 raft_consensus.cc:482] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:26.302560 15526 raft_consensus.cc:504] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.304330 15526 leader_election.cc:288] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e0c3857b25d465bb079b79f071870b8 (127.9.194.195:35993), ef38c40b5a424d7b95b405ba827c9d76 (127.9.194.194:37111)
W20260807 22:50:26.307070 15362 tablet.cc:2369] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:26.308122 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.304778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:59005 (local address 127.9.194.194:37111)
0807 22:50:26.304994 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.305001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.305933 (+ 932us) server_negotiation.cc:407] Connection header received
0807 22:50:26.306070 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.306076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.306147 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.306236 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.307418 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.307451 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.307457 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.307476 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.307520 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.307533 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.307658 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.307749 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.307816 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:50:26.308362 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.304950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:46127 (local address 127.9.194.195:35993)
0807 22:50:26.305162 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.305171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.306042 (+ 871us) server_negotiation.cc:407] Connection header received
0807 22:50:26.306167 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.306170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.306241 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.306339 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.307651 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.307656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.307660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.307681 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.307722 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.307736 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.307852 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.308042 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.308104 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:50:26.308547 15529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.304950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:37111 (local address 127.9.194.193:59005)
0807 22:50:26.305814 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.305831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.305938 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.306293 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.306300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.306328 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.307178 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.307183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.307204 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.307258 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.307645 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.307649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.307653 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.307788 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.307848 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":494,"threads_started":1}
I20260807 22:50:26.308651 15530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.304987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:35993 (local address 127.9.194.193:46127)
0807 22:50:26.305919 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.305937 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:26.306056 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:26.306394 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:26.306397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:26.306418 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:26.307400 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:50:26.307406 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:26.307450 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:50:26.307497 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:26.307911 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:26.307918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:26.307924 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:50:26.308074 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:50:26.308127 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":484,"threads_started":1}
I20260807 22:50:26.309480 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6c2f848904afd9426789642b1f008" candidate_uuid: "2277985909fd445f8845d4b7b2963204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef38c40b5a424d7b95b405ba827c9d76" is_pre_election: true
I20260807 22:50:26.309556 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6c2f848904afd9426789642b1f008" candidate_uuid: "2277985909fd445f8845d4b7b2963204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0c3857b25d465bb079b79f071870b8" is_pre_election: true
I20260807 22:50:26.309999 15311 raft_consensus.cc:2454] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2277985909fd445f8845d4b7b2963204 in term 0.
I20260807 22:50:26.310122 15452 raft_consensus.cc:2454] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2277985909fd445f8845d4b7b2963204 in term 0.
I20260807 22:50:26.310710 15098 leader_election.cc:302] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2277985909fd445f8845d4b7b2963204, ef38c40b5a424d7b95b405ba827c9d76; no voters:
I20260807 22:50:26.311128 15526 raft_consensus.cc:2781] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:26.311275 15526 raft_consensus.cc:482] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:26.311376 15526 raft_consensus.cc:3037] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:26.313614 15526 raft_consensus.cc:504] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.314224 15526 leader_election.cc:288] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [CANDIDATE]: Term 1 election: Requested vote from peers 2e0c3857b25d465bb079b79f071870b8 (127.9.194.195:35993), ef38c40b5a424d7b95b405ba827c9d76 (127.9.194.194:37111)
I20260807 22:50:26.314685 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6c2f848904afd9426789642b1f008" candidate_uuid: "2277985909fd445f8845d4b7b2963204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef38c40b5a424d7b95b405ba827c9d76"
I20260807 22:50:26.314721 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6c2f848904afd9426789642b1f008" candidate_uuid: "2277985909fd445f8845d4b7b2963204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0c3857b25d465bb079b79f071870b8"
I20260807 22:50:26.314898 15311 raft_consensus.cc:3037] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:26.314929 15452 raft_consensus.cc:3037] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:26.317242 15452 raft_consensus.cc:2454] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2277985909fd445f8845d4b7b2963204 in term 1.
I20260807 22:50:26.317245 15311 raft_consensus.cc:2454] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2277985909fd445f8845d4b7b2963204 in term 1.
I20260807 22:50:26.317639 15100 leader_election.cc:302] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2277985909fd445f8845d4b7b2963204, 2e0c3857b25d465bb079b79f071870b8; no voters:
I20260807 22:50:26.317953 15526 raft_consensus.cc:2781] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:26.318482 15526 raft_consensus.cc:686] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 1 LEADER]: Becoming Leader. State: Replica: 2277985909fd445f8845d4b7b2963204, State: Running, Role: LEADER
I20260807 22:50:26.319020 15526 consensus_queue.cc:237] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } }
I20260807 22:50:26.324141 15015 catalog_manager.cc:5495] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2277985909fd445f8845d4b7b2963204 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "2277985909fd445f8845d4b7b2963204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2277985909fd445f8845d4b7b2963204" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 38621 } health_report { overall_health: HEALTHY } } }
I20260807 22:50:26.367996 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.365879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.9.194.193:38621)
0807 22:50:26.366106 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.366111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.366151 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:50:26.366241 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.366244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.366323 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.366459 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.367351 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.367355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.367357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.367382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.367411 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.367470 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.367559 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.367648 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.367695 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:50:26.376272 15311 raft_consensus.cc:1261] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 1 FOLLOWER]: Refusing update from remote peer 2277985909fd445f8845d4b7b2963204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:26.376272 15452 raft_consensus.cc:1261] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 1 FOLLOWER]: Refusing update from remote peer 2277985909fd445f8845d4b7b2963204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:26.377434 15531 consensus_queue.cc:1035] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef38c40b5a424d7b95b405ba827c9d76" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:26.377781 15526 consensus_queue.cc:1035] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0c3857b25d465bb079b79f071870b8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:26.389236 15534 mvcc.cc:205] Tried to move back new op lower bound from 7316041836026818560 to 7316041835807207424. Current Snapshot: MvccSnapshot[applied={T|T < 7316041836026818560}]
I20260807 22:50:26.405285 15014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:26.405572 15014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:26.409512 15014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b26897c71a3b4ea5a4ff0b5df854fccd: Sending DeleteTablet for 3 replicas of tablet 5ec6c2f848904afd9426789642b1f008
I20260807 22:50:26.410812 15431 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6c2f848904afd9426789642b1f008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:26 UTC) from {username='slave'} at 127.0.0.1:51840
I20260807 22:50:26.410812 15291 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6c2f848904afd9426789642b1f008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:26 UTC) from {username='slave'} at 127.0.0.1:59750
I20260807 22:50:26.410869 15149 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6c2f848904afd9426789642b1f008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:26 UTC) from {username='slave'} at 127.0.0.1:42912
I20260807 22:50:26.415071 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15082
I20260807 22:50:26.422732 15543 tablet_replica.cc:330] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: stopping tablet replica
I20260807 22:50:26.423242 15543 raft_consensus.cc:2229] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:26.424073 15543 raft_consensus.cc:2258] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:26.424947 15544 tablet_replica.cc:330] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: stopping tablet replica
I20260807 22:50:26.425433 15544 raft_consensus.cc:2229] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:26.425951 15544 raft_consensus.cc:2258] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:26.426999 15545 tablet_replica.cc:330] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: stopping tablet replica
I20260807 22:50:26.427481 15545 raft_consensus.cc:2229] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:26.427973 15545 raft_consensus.cc:2258] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:26.428113 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.416999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.9.194.193:38621)
0807 22:50:26.417253 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.417262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.425250 (+ 7988us) server_negotiation.cc:407] Connection header received
0807 22:50:26.425617 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.425627 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.425715 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.425815 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.427234 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.427240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.427244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.427266 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.427303 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.427318 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.427418 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.427708 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.427773 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 22:50:26.429153 15543 ts_tablet_manager.cc:1905] T 5ec6c2f848904afd9426789642b1f008 P 2277985909fd445f8845d4b7b2963204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:26.429288 15215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:26.430423 15545 ts_tablet_manager.cc:1905] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:26.431197 15544 ts_tablet_manager.cc:1905] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:26.438320 15544 ts_tablet_manager.cc:1918] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:26.438512 15544 log.cc:1196] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/5ec6c2f848904afd9426789642b1f008
I20260807 22:50:26.439069 15544 ts_tablet_manager.cc:1939] T 5ec6c2f848904afd9426789642b1f008 P 2e0c3857b25d465bb079b79f071870b8: Deleting consensus metadata
I20260807 22:50:26.439502 15545 ts_tablet_manager.cc:1918] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:26.439687 15545 log.cc:1196] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/5ec6c2f848904afd9426789642b1f008
I20260807 22:50:26.440213 15545 ts_tablet_manager.cc:1939] T 5ec6c2f848904afd9426789642b1f008 P ef38c40b5a424d7b95b405ba827c9d76: Deleting consensus metadata
I20260807 22:50:26.441278 15001 catalog_manager.cc:4841] TS 2e0c3857b25d465bb079b79f071870b8 (127.9.194.195:35993): tablet 5ec6c2f848904afd9426789642b1f008 (table test-table [id=d566231b81c449d4b005b011252670aa]) successfully deleted
I20260807 22:50:26.441929 14999 catalog_manager.cc:4841] TS ef38c40b5a424d7b95b405ba827c9d76 (127.9.194.194:37111): tablet 5ec6c2f848904afd9426789642b1f008 (table test-table [id=d566231b81c449d4b005b011252670aa]) successfully deleted
I20260807 22:50:26.527931 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15082
I20260807 22:50:26.547787 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15224
W20260807 22:50:26.547843 15000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:38621 (error 108)
I20260807 22:50:26.549743 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.549235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42946)
0807 22:50:26.549408 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.549604 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 22:50:26.550127 15000 catalog_manager.cc:4584] TS 2277985909fd445f8845d4b7b2963204 (127.9.194.193:38621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ec6c2f848904afd9426789642b1f008: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
I20260807 22:50:26.550972 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.548604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.9.194.194:37111)
0807 22:50:26.548774 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.548780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.548828 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:50:26.549097 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.549104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.549214 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.549312 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.550320 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.550329 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.550334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.550354 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.550400 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.550421 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.550551 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.550639 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.550698 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:50:26.551659 15356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:26.603775 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.603278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42954)
0807 22:50:26.603535 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.603627 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 22:50:26.604820 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.604457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42962)
0807 22:50:26.604604 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.604694 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 22:50:26.639884 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15224
I20260807 22:50:26.658141 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15365
I20260807 22:50:26.661022 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.659016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.9.194.195:35993)
0807 22:50:26.659191 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:26.659196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:26.659216 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:50:26.659355 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:26.659359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:26.659533 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:26.659619 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:26.660493 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:26.660498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:26.660501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:26.660515 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:26.660548 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:26.660560 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:26.660668 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:26.660739 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 22:50:26.660788 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:50:26.661653 15496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:26.666036 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.665561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42968)
0807 22:50:26.665722 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.665847 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:50:26.667186 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.666770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42970)
0807 22:50:26.666916 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.667007 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 22:50:26.747802 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.747156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42974)
0807 22:50:26.747452 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.747595 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 22:50:26.749012 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:26.748559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:38621 (local address 127.0.0.1:42984)
0807 22:50:26.748779 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:26.748865 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:38621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 22:50:26.752575 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15365
I20260807 22:50:26.771538 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 14983
I20260807 22:50:26.772801 15051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:26.883407 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 14983
2026-08-07T22:50:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2792 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 22:50:26.928117 9995 mini_oidc.cc:152] Starting JWKS server
I20260807 22:50:26.931059 9995 webserver.cc:460] Webserver started at https://127.0.0.1:38139/ using document root <none> and password file <none>
I20260807 22:50:26.931465 9995 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:50:26.932143 9995 webserver.cc:460] Webserver started at http://127.0.0.1:37419/ using document root <none> and password file <none>
2026-08-07T22:50:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:26Z Disabled control of system clock
I20260807 22:50:26.964717 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40065
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:40065
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38139/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/testchainca.cert with env {}
W20260807 22:50:27.291298 15558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:27.291658 15558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:27.291814 15558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:27.298867 15558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:27.298971 15558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:27.299010 15558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:27.299067 15558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:27.299110 15558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:50:27.299165 15558 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:50:27.299204 15558 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38139/jwks.json
I20260807 22:50:27.308848 15558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:40065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38139/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:27.310057 15558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:27.311285 15558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:27.323168 15564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:27.323136 15567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:27.323200 15565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:27.324224 15558 server_base.cc:1034] running on GCE node
I20260807 22:50:27.324936 15558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:27.326197 15558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:27.327461 15558 hybrid_clock.cc:648] HybridClock initialized: now 1786143027327392 us; error 74 us; skew 500 ppm
I20260807 22:50:27.327809 15558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:27.330052 15558 webserver.cc:460] Webserver started at http://127.9.194.254:44283/ using document root <none> and password file <none>
I20260807 22:50:27.330721 15558 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:27.330854 15558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:27.331175 15558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:27.334175 15558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "a939bad81295447aa12baac914f630ad"
format_stamp: "Formatted at 2026-08-07 22:50:27 on dist-test-slave-5k54"
server_key: "418db43a4b37d599cc0b47565402361d"
server_key_iv: "e959decac5fb47bb3b8e31a120cf8578"
server_key_version: "encryptionkey@0"
I20260807 22:50:27.334955 15558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "a939bad81295447aa12baac914f630ad"
format_stamp: "Formatted at 2026-08-07 22:50:27 on dist-test-slave-5k54"
server_key: "418db43a4b37d599cc0b47565402361d"
server_key_iv: "e959decac5fb47bb3b8e31a120cf8578"
server_key_version: "encryptionkey@0"
I20260807 22:50:27.339939 15558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:50:27.343293 15574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:27.344707 15558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:50:27.344902 15558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "a939bad81295447aa12baac914f630ad"
format_stamp: "Formatted at 2026-08-07 22:50:27 on dist-test-slave-5k54"
server_key: "418db43a4b37d599cc0b47565402361d"
server_key_iv: "e959decac5fb47bb3b8e31a120cf8578"
server_key_version: "encryptionkey@0"
I20260807 22:50:27.345117 15558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:27.365249 15558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:27.366148 15558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:27.366473 15558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:27.385248 15558 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:40065
I20260807 22:50:27.385283 15626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:40065 every 8 connection(s)
I20260807 22:50:27.386580 15558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:27.390385 15627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:27.393868 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15558
I20260807 22:50:27.394105 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:27.394953 9995 external_mini_cluster.cc:1428] Setting key 6ba79e10611dffb3e6216d7c7e281c37
I20260807 22:50:27.402848 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad: Bootstrap starting.
I20260807 22:50:27.405928 15627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:27.406970 15627 log.cc:824] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:27.408317 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:27.396916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.9.194.254:40065)
0807 22:50:27.397828 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:27.397838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:27.397879 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:27.399600 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:27.399621 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:27.400087 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:27.400483 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:27.401107 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.402346 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.403800 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.404483 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.404698 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:27.405866 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:27.405875 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:27.405889 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:27.406467 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:27.406554 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:27.406574 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:27.406696 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:27.406916 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 22:50:27.407239 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":323,"threads_started":1}
I20260807 22:50:27.409865 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad: No bootstrap required, opened a new log
I20260807 22:50:27.411957 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:50:27.417492 15627 raft_consensus.cc:348] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } }
I20260807 22:50:27.417838 15627 raft_consensus.cc:374] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:27.417996 15627 raft_consensus.cc:729] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a939bad81295447aa12baac914f630ad, State: Initialized, Role: FOLLOWER
I20260807 22:50:27.418546 15627 consensus_queue.cc:260] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } }
I20260807 22:50:27.418772 15627 raft_consensus.cc:388] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:27.418917 15627 raft_consensus.cc:482] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:27.419090 15627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:27.421625 15627 raft_consensus.cc:504] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } }
I20260807 22:50:27.422132 15627 leader_election.cc:302] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a939bad81295447aa12baac914f630ad; no voters:
I20260807 22:50:27.422669 15627 leader_election.cc:288] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:27.422830 15634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:27.423523 15634 raft_consensus.cc:686] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [term 1 LEADER]: Becoming Leader. State: Replica: a939bad81295447aa12baac914f630ad, State: Running, Role: LEADER
I20260807 22:50:27.424016 15634 consensus_queue.cc:237] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } }
I20260807 22:50:27.424778 15627 sys_catalog.cc:564] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:27.426599 15635 sys_catalog.cc:455] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a939bad81295447aa12baac914f630ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } } }
I20260807 22:50:27.426802 15636 sys_catalog.cc:455] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a939bad81295447aa12baac914f630ad. Latest consensus state: current_term: 1 leader_uuid: "a939bad81295447aa12baac914f630ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a939bad81295447aa12baac914f630ad" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40065 } } }
I20260807 22:50:27.427049 15635 sys_catalog.cc:458] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:27.427074 15636 sys_catalog.cc:458] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:27.428079 15639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:27.433373 15639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:27.441535 15639 catalog_manager.cc:1347] Generated new cluster ID: 960c8fc6db7147d894c71ca291057924
I20260807 22:50:27.441653 15639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:27.450718 15639 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:27.452561 15639 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:27.462586 15639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a939bad81295447aa12baac914f630ad: Generated new TSK 0
I20260807 22:50:27.463637 15639 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:50:27.501009 15661 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 22:50:27.501253 15661 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:50:27.503496 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:27.480434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.9.194.254:40065)
0807 22:50:27.480633 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:27.480642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:27.481370 (+ 728us) server_negotiation.cc:407] Connection header received
0807 22:50:27.481754 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:27.481758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:27.481836 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:27.481955 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:50:27.482597 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.483312 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.485429 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.485880 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.485984 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:27.486601 (+ 617us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:50:27.502874 (+ 16273us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:50:27.503179 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52904: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 22:50:27.503644 15630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52904: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 22:50:27.505623 15660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:27.480448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:40065 (local address 127.0.0.1:52904)
0807 22:50:27.481257 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:27.481278 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:27.481470 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:27.482013 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:27.482018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:27.482032 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:50:27.482431 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:27.482445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.483489 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.483494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:27.485272 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:27.485287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:27.486070 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:27.486077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:27.486271 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:27.486348 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:50:27.503179 (+ 16831us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:50:27.503558 (+ 379us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.194.254:40065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":367,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.194.254:40065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 22:50:27.507565 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15558
I20260807 22:50:27.508728 15622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:27.602794 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15558
2026-08-07T22:50:27Z chronyd exiting
I20260807 22:50:27.655231 9995 test_util.cc:175] -----------------------------------------------
I20260807 22:50:27.655413 9995 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142983724342-9995-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 22:50:27.659260 9995 mini_oidc.cc:152] Starting JWKS server
I20260807 22:50:27.661165 9995 webserver.cc:460] Webserver started at https://127.0.0.1:40219/ using document root <none> and password file <none>
I20260807 22:50:27.661412 9995 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:50:27.662096 9995 webserver.cc:460] Webserver started at http://127.0.0.1:33889/ using document root <none> and password file <none>
2026-08-07T22:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:27Z Disabled control of system clock
I20260807 22:50:27.693914 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33475
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33475
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40219/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/testchainca.cert with env {}
W20260807 22:50:28.015586 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:28.015940 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:28.016098 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:28.023202 15669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:28.023350 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:28.023458 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:28.023576 15669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:28.023660 15669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:50:28.023752 15669 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:50:28.023833 15669 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40219/jwks.json
I20260807 22:50:28.033483 15669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40219/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:28.034783 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:28.036137 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:28.047575 15674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:28.047720 15675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:28.048192 15677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:28.048254 15669 server_base.cc:1034] running on GCE node
I20260807 22:50:28.049013 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:28.050379 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:28.051605 15669 hybrid_clock.cc:648] HybridClock initialized: now 1786143028051558 us; error 43 us; skew 500 ppm
I20260807 22:50:28.051955 15669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:28.057875 15669 webserver.cc:460] Webserver started at http://127.9.194.254:40689/ using document root <none> and password file <none>
I20260807 22:50:28.058610 15669 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:28.058751 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:28.059090 15669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:28.062031 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "d221fd5d65054eaf8ae728235b89d8b8"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "ce6fca348214b0e8d511c0daa0c4c73d"
server_key_iv: "6c5753a2155bd9d95445d61fc7038023"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.062826 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "d221fd5d65054eaf8ae728235b89d8b8"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "ce6fca348214b0e8d511c0daa0c4c73d"
server_key_iv: "6c5753a2155bd9d95445d61fc7038023"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.067940 15669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:50:28.071255 15684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:28.072790 15669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:50:28.072988 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d221fd5d65054eaf8ae728235b89d8b8"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "ce6fca348214b0e8d511c0daa0c4c73d"
server_key_iv: "6c5753a2155bd9d95445d61fc7038023"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.073199 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:28.098449 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:28.099342 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:28.099691 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:28.118252 15669 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33475
I20260807 22:50:28.118273 15738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33475 every 8 connection(s)
I20260807 22:50:28.119635 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:28.122797 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15669
I20260807 22:50:28.123008 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:28.123838 15740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:28.123919 9995 external_mini_cluster.cc:1428] Setting key e445e01ea83e9ac2ff3beaf08aeeed17
I20260807 22:50:28.137507 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:28.125806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.9.194.254:33475)
0807 22:50:28.126775 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:28.126789 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:28.126836 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:50:28.128615 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:28.128636 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:28.129107 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:28.129442 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:28.130032 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.131399 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.133192 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.133816 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.133972 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:28.135215 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:28.135224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:28.135237 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:28.135834 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:28.135924 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:28.135944 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:28.136043 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:28.136269 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:50:28.136593 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":351,"threads_started":1}
I20260807 22:50:28.137792 15740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8: Bootstrap starting.
I20260807 22:50:28.141091 15740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:28.141206 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:50:28.142088 15740 log.cc:824] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:28.144941 15740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8: No bootstrap required, opened a new log
I20260807 22:50:28.152300 15740 raft_consensus.cc:348] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } }
I20260807 22:50:28.152654 15740 raft_consensus.cc:374] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:28.152748 15740 raft_consensus.cc:729] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d221fd5d65054eaf8ae728235b89d8b8, State: Initialized, Role: FOLLOWER
I20260807 22:50:28.153342 15740 consensus_queue.cc:260] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } }
I20260807 22:50:28.153586 15740 raft_consensus.cc:388] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:28.153753 15740 raft_consensus.cc:482] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:28.153920 15740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:28.156149 15740 raft_consensus.cc:504] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } }
I20260807 22:50:28.156632 15740 leader_election.cc:302] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d221fd5d65054eaf8ae728235b89d8b8; no voters:
I20260807 22:50:28.157194 15740 leader_election.cc:288] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:28.157351 15746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:28.158106 15746 raft_consensus.cc:686] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [term 1 LEADER]: Becoming Leader. State: Replica: d221fd5d65054eaf8ae728235b89d8b8, State: Running, Role: LEADER
I20260807 22:50:28.158667 15746 consensus_queue.cc:237] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } }
I20260807 22:50:28.159392 15740 sys_catalog.cc:564] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:28.161438 15748 sys_catalog.cc:455] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d221fd5d65054eaf8ae728235b89d8b8. Latest consensus state: current_term: 1 leader_uuid: "d221fd5d65054eaf8ae728235b89d8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } } }
I20260807 22:50:28.161830 15748 sys_catalog.cc:458] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:28.165160 15747 sys_catalog.cc:455] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d221fd5d65054eaf8ae728235b89d8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d221fd5d65054eaf8ae728235b89d8b8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33475 } } }
I20260807 22:50:28.165649 15747 sys_catalog.cc:458] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:28.167722 15751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:28.173146 15751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:28.181855 15751 catalog_manager.cc:1347] Generated new cluster ID: 4fafe920c7464c71a11a9e000cc5c36c
I20260807 22:50:28.182009 15751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:28.239972 15751 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:28.241842 15751 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:28.251483 15751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d221fd5d65054eaf8ae728235b89d8b8: Generated new TSK 0
I20260807 22:50:28.252348 15751 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:50:28.296708 15773 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 22:50:28.296841 15773 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:50:28.297976 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:28.276853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.9.194.254:33475)
0807 22:50:28.277079 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:28.277089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:28.277589 (+ 500us) server_negotiation.cc:407] Connection header received
0807 22:50:28.277841 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:28.277846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:28.278007 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:28.278128 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:50:28.278685 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.279409 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.281430 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.281864 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.281964 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:28.282521 (+ 557us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:50:28.297441 (+ 14920us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:50:28.297732 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 22:50:28.298125 15743 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 22:50:28.298659 15772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:28.276843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33475 (local address 127.0.0.1:36202)
0807 22:50:28.277480 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:28.277506 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:28.277655 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:28.278169 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:28.278175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:28.278188 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:50:28.278512 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:28.278524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.279606 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.279612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:28.281294 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:28.281310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.282030 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.282037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:28.282245 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:28.282290 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:50:28.297662 (+ 15372us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:50:28.297939 (+ 277us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.194.254:33475: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":324,"threads_started":1}
I20260807 22:50:28.300209 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15669
I20260807 22:50:28.301429 15734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:28.399351 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15669
2026-08-07T22:50:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (798 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 22:50:28.457508 9995 mini_oidc.cc:152] Starting JWKS server
I20260807 22:50:28.459404 9995 webserver.cc:460] Webserver started at https://127.0.0.1:46303/ using document root <none> and password file <none>
I20260807 22:50:28.459733 9995 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:50:28.460353 9995 webserver.cc:460] Webserver started at http://127.0.0.1:34385/ using document root <none> and password file <none>
2026-08-07T22:50:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:28Z Disabled control of system clock
I20260807 22:50:28.491657 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45397
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:36239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:45397
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46303/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:50:28.813902 15780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:28.814251 15780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:28.814417 15780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:28.821501 15780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:28.821660 15780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:28.821758 15780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:28.821869 15780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:28.821957 15780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:50:28.822047 15780 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:50:28.822124 15780 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46303/jwks.json
I20260807 22:50:28.831215 15780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:36239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:45397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46303/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:28.832628 15780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:28.834507 15780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:28.847339 15785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:28.847365 15786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:28.847476 15789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:28.848093 15780 server_base.cc:1034] running on GCE node
I20260807 22:50:28.848979 15780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:28.850484 15780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:28.851759 15780 hybrid_clock.cc:648] HybridClock initialized: now 1786143028851688 us; error 66 us; skew 500 ppm
I20260807 22:50:28.852166 15780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:28.854628 15780 webserver.cc:460] Webserver started at http://127.9.194.254:41769/ using document root <none> and password file <none>
I20260807 22:50:28.855360 15780 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:28.855527 15780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:28.855872 15780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:28.858870 15780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "224b36633a1b4b0d8ab21da2a055c872"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "bca1f92d3b1340d0703f05491ed6aaa6"
server_key_iv: "c6b6b40619230e13f1749bf1335e31f6"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.859771 15780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "224b36633a1b4b0d8ab21da2a055c872"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "bca1f92d3b1340d0703f05491ed6aaa6"
server_key_iv: "c6b6b40619230e13f1749bf1335e31f6"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.865322 15780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:28.868857 15796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:28.870855 15780 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:50:28.871078 15780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "224b36633a1b4b0d8ab21da2a055c872"
format_stamp: "Formatted at 2026-08-07 22:50:28 on dist-test-slave-5k54"
server_key: "bca1f92d3b1340d0703f05491ed6aaa6"
server_key_iv: "c6b6b40619230e13f1749bf1335e31f6"
server_key_version: "encryptionkey@0"
I20260807 22:50:28.871398 15780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:28.891587 15780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:28.892364 15780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:28.892635 15780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:28.911204 15780 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:45397
I20260807 22:50:28.911223 15849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:45397 every 8 connection(s)
I20260807 22:50:28.912573 15780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:28.916299 15850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:28.919656 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15780
I20260807 22:50:28.919884 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:28.920710 9995 external_mini_cluster.cc:1428] Setting key 968bd30711396afa5a152f6334fc808c
I20260807 22:50:28.928639 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872: Bootstrap starting.
I20260807 22:50:28.931632 15850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:28.932677 15850 log.cc:824] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:28.934209 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:28.922662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.9.194.254:45397)
0807 22:50:28.923632 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:28.923644 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:28.923685 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:28.925317 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:28.925338 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:28.925808 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:28.926210 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:28.926809 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.928103 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.929460 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:28.930151 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:28.930370 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:28.931556 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:28.931566 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:28.931580 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:28.932175 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:28.932270 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:28.932291 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:28.932424 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:28.932649 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 22:50:28.932954 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":323,"threads_started":1}
I20260807 22:50:28.935977 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872: No bootstrap required, opened a new log
I20260807 22:50:28.937995 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:50:28.943691 15850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } }
I20260807 22:50:28.944061 15850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:28.944169 15850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224b36633a1b4b0d8ab21da2a055c872, State: Initialized, Role: FOLLOWER
I20260807 22:50:28.944712 15850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } }
I20260807 22:50:28.945008 15850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:28.945175 15850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:28.945370 15850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:28.947484 15850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } }
I20260807 22:50:28.947979 15850 leader_election.cc:302] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 224b36633a1b4b0d8ab21da2a055c872; no voters:
I20260807 22:50:28.948544 15850 leader_election.cc:288] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:28.948717 15856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:28.949445 15856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [term 1 LEADER]: Becoming Leader. State: Replica: 224b36633a1b4b0d8ab21da2a055c872, State: Running, Role: LEADER
I20260807 22:50:28.950086 15856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } }
I20260807 22:50:28.950691 15850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:28.953191 15859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "224b36633a1b4b0d8ab21da2a055c872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } } }
I20260807 22:50:28.953699 15859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:28.953161 15860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 224b36633a1b4b0d8ab21da2a055c872. Latest consensus state: current_term: 1 leader_uuid: "224b36633a1b4b0d8ab21da2a055c872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224b36633a1b4b0d8ab21da2a055c872" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45397 } } }
I20260807 22:50:28.953820 15860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:28.954553 15866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:28.962967 15866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:28.971808 15866 catalog_manager.cc:1347] Generated new cluster ID: e00e2445b0ec477cae87220c558a358e
I20260807 22:50:28.971961 15866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:28.982033 15866 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:28.983829 15866 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:28.993079 15866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 224b36633a1b4b0d8ab21da2a055c872: Generated new TSK 0
I20260807 22:50:28.994059 15866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:50:29.017699 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:29.008915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.9.194.254:45397)
0807 22:50:29.009156 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:29.009163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:29.009607 (+ 444us) server_negotiation.cc:407] Connection header received
0807 22:50:29.009852 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:29.009855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:29.009938 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:29.010095 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:50:29.010671 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.011489 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.013520 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.013948 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.014040 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:29.014645 (+ 605us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:50:29.017069 (+ 2424us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:50:29.017396 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59312: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 22:50:29.017822 15853 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59312: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 22:50:29.019872 15883 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:29.008789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45397 (local address 127.0.0.1:59312)
0807 22:50:29.009488 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:29.009509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:29.009670 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:29.010143 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:29.010148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:29.010162 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:50:29.010512 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:29.010525 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.011634 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.011640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:29.013370 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:29.013385 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.014141 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.014148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:29.014345 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:29.014396 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:50:29.017291 (+ 2895us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:50:29.019382 (+ 2091us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.194.254:45397: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":1759,"thread_start_us":336,"threads_started":1}
I20260807 22:50:29.020855 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15780
I20260807 22:50:29.022027 15844 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 22:50:29.029580 15884 webserver.cc:533] Webserver: sslize failed:
E20260807 22:50:29.029695 15884 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:50:29.111949 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15780
2026-08-07T22:50:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (711 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T22:50:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:29Z Disabled control of system clock
I20260807 22:50:29.201409 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:34597
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:34597
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/testchainca.cert with env {}
W20260807 22:50:29.528714 15890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:29.529027 15890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:29.529204 15890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:29.536327 15890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:29.536464 15890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:29.536559 15890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:29.536669 15890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:29.536746 15890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:50:29.536846 15890 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:50:29.536921 15890 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 22:50:29.546713 15890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:34597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:34597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:29.548058 15890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:29.549364 15890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:29.561604 15895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:29.561583 15896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:29.562331 15898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:29.562302 15890 server_base.cc:1034] running on GCE node
I20260807 22:50:29.563205 15890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:29.564589 15890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:29.565788 15890 hybrid_clock.cc:648] HybridClock initialized: now 1786143029565729 us; error 58 us; skew 500 ppm
I20260807 22:50:29.566138 15890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:50:29.568465 15890 webserver.cc:460] Webserver started at http://127.9.194.254:37121/ using document root <none> and password file <none>
I20260807 22:50:29.569172 15890 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:29.569291 15890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:29.569629 15890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:29.572537 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "a620fba0e8e04822a9182da082b7764d"
format_stamp: "Formatted at 2026-08-07 22:50:29 on dist-test-slave-5k54"
server_key: "5431d42fd248c997d3931425f4f98d08"
server_key_iv: "d9c3bb60e94a32e21b95994932a7afa8"
server_key_version: "encryptionkey@0"
I20260807 22:50:29.573341 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "a620fba0e8e04822a9182da082b7764d"
format_stamp: "Formatted at 2026-08-07 22:50:29 on dist-test-slave-5k54"
server_key: "5431d42fd248c997d3931425f4f98d08"
server_key_iv: "d9c3bb60e94a32e21b95994932a7afa8"
server_key_version: "encryptionkey@0"
I20260807 22:50:29.578326 15890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:50:29.581696 15904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:29.583118 15890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:29.583333 15890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "a620fba0e8e04822a9182da082b7764d"
format_stamp: "Formatted at 2026-08-07 22:50:29 on dist-test-slave-5k54"
server_key: "5431d42fd248c997d3931425f4f98d08"
server_key_iv: "d9c3bb60e94a32e21b95994932a7afa8"
server_key_version: "encryptionkey@0"
I20260807 22:50:29.583635 15890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:29.599749 15890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:29.600584 15890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:29.600876 15890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:29.619066 15890 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:34597
I20260807 22:50:29.619091 15961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:34597 every 8 connection(s)
I20260807 22:50:29.620471 15890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:29.621057 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 15890
I20260807 22:50:29.621279 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:29.622076 9995 external_mini_cluster.cc:1428] Setting key 7e1bfe05f862e3bdf9b93e0fded3a722
I20260807 22:50:29.625102 15962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:29.635286 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:29.624042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45836 (local address 127.9.194.254:34597)
0807 22:50:29.625008 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:29.625019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:29.625060 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:29.626725 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:29.626744 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:29.627218 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:29.627676 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:29.628247 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.629556 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.630883 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.631603 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.631795 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:29.633016 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:29.633023 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:29.633039 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:50:29.633609 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:29.633681 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:50:29.633704 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:29.633859 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:29.634074 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 22:50:29.634364 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":327,"threads_started":1}
I20260807 22:50:29.635901 15962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d: Bootstrap starting.
I20260807 22:50:29.638947 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:50:29.639164 15962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:29.640208 15962 log.cc:824] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:29.643006 15962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d: No bootstrap required, opened a new log
I20260807 22:50:29.650098 15962 raft_consensus.cc:348] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } }
I20260807 22:50:29.650470 15962 raft_consensus.cc:374] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:29.650595 15962 raft_consensus.cc:729] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a620fba0e8e04822a9182da082b7764d, State: Initialized, Role: FOLLOWER
I20260807 22:50:29.651233 15962 consensus_queue.cc:260] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } }
I20260807 22:50:29.651562 15962 raft_consensus.cc:388] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:29.651711 15962 raft_consensus.cc:482] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:29.651892 15962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:29.653945 15962 raft_consensus.cc:504] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } }
I20260807 22:50:29.654444 15962 leader_election.cc:302] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a620fba0e8e04822a9182da082b7764d; no voters:
I20260807 22:50:29.655016 15962 leader_election.cc:288] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:29.655174 15967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:29.655916 15967 raft_consensus.cc:686] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [term 1 LEADER]: Becoming Leader. State: Replica: a620fba0e8e04822a9182da082b7764d, State: Running, Role: LEADER
I20260807 22:50:29.656451 15967 consensus_queue.cc:237] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } }
I20260807 22:50:29.657087 15962 sys_catalog.cc:564] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:29.659722 15969 sys_catalog.cc:455] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a620fba0e8e04822a9182da082b7764d. Latest consensus state: current_term: 1 leader_uuid: "a620fba0e8e04822a9182da082b7764d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } } }
I20260807 22:50:29.660391 15969 sys_catalog.cc:458] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:29.661386 15975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:29.661293 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a620fba0e8e04822a9182da082b7764d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a620fba0e8e04822a9182da082b7764d" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34597 } } }
I20260807 22:50:29.662755 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:29.667023 15975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:29.675717 15975 catalog_manager.cc:1347] Generated new cluster ID: b478b3ff3557461ba568dd1cb6062b6c
I20260807 22:50:29.675830 15975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:29.702090 15975 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:29.703387 15975 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:29.714919 15975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a620fba0e8e04822a9182da082b7764d: Generated new TSK 0
I20260807 22:50:29.715983 15975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:50:29.783238 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:29.730281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.9.194.254:34597)
0807 22:50:29.730467 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:29.730474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:29.730796 (+ 322us) server_negotiation.cc:407] Connection header received
0807 22:50:29.730991 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:29.730996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:29.731095 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:29.731254 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:50:29.731720 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.732845 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.734212 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.734912 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.735062 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:29.735533 (+ 471us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:50:29.782364 (+ 46831us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:50:29.782901 (+ 537us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45838: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 22:50:29.783398 15965 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45838: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 22:50:29.783521 15991 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:29.730113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:34597 (local address 127.0.0.1:45838)
0807 22:50:29.730694 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:29.730710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:29.730824 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:29.731268 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:29.731271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:29.731280 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:50:29.731581 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:29.731590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.732985 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.732988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:29.734078 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:29.734089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:29.735090 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:29.735095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:29.735215 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:29.735245 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:50:29.782748 (+ 47503us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:50:29.783050 (+ 302us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.194.254:34597: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":322,"threads_started":1}
I20260807 22:50:29.784924 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 15890
I20260807 22:50:29.786190 15957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:29.883041 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 15890
2026-08-07T22:50:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (748 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (412 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (413 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:30 dist-test-slave-5k54 krb5kdc[16008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:30 dist-test-slave-5k54 krb5kdc[16008](info): set up 2 sockets
Aug 07 22:50:30 dist-test-slave-5k54 krb5kdc[16008](info): commencing operation
krb5kdc: starting...
W20260807 22:50:31.857105 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:32 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:32.095381 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33533
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:32.429260 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:32.429571 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:32.429754 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:32.437224 16021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:32.437335 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:32.437448 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:32.437548 16021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:32.437635 16021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:32.448477 16021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:32.449922 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:32.451267 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:32.463205 16029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:32.463348 16026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:32.463318 16027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:32.464277 16021 server_base.cc:1034] running on GCE node
I20260807 22:50:32.465032 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:32.466373 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:32.467532 16021 hybrid_clock.cc:648] HybridClock initialized: now 1786143032467472 us; error 39 us; skew 500 ppm
I20260807 22:50:32.467835 16021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:32 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143032, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:32.470968 16021 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:32.472537 16021 webserver.cc:460] Webserver started at http://127.9.194.254:38457/ using document root <none> and password file <none>
I20260807 22:50:32.473166 16021 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:32.473268 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:32.473556 16021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:32.476706 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "f7e2f4020a754f109889696b5202d12a"
format_stamp: "Formatted at 2026-08-07 22:50:32 on dist-test-slave-5k54"
server_key: "2bd931781b69ad45ee298600297e7b7d"
server_key_iv: "ccf3152b5334c441f4b50a0cd19bde53"
server_key_version: "encryptionkey@0"
I20260807 22:50:32.477464 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "f7e2f4020a754f109889696b5202d12a"
format_stamp: "Formatted at 2026-08-07 22:50:32 on dist-test-slave-5k54"
server_key: "2bd931781b69ad45ee298600297e7b7d"
server_key_iv: "ccf3152b5334c441f4b50a0cd19bde53"
server_key_version: "encryptionkey@0"
I20260807 22:50:32.482540 16021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:50:32.485906 16037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:32.487167 16021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:32.487303 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "f7e2f4020a754f109889696b5202d12a"
format_stamp: "Formatted at 2026-08-07 22:50:32 on dist-test-slave-5k54"
server_key: "2bd931781b69ad45ee298600297e7b7d"
server_key_iv: "ccf3152b5334c441f4b50a0cd19bde53"
server_key_version: "encryptionkey@0"
I20260807 22:50:32.487519 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:32.514461 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:32.518527 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:32.518776 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:32.537456 16021 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33533
I20260807 22:50:32.537480 16091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33533 every 8 connection(s)
I20260807 22:50:32.538906 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:32.542619 16092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:32.548604 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16021
I20260807 22:50:32.548841 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:32.549727 9995 external_mini_cluster.cc:1428] Setting key 01f31b523143876fc403ac2a03545157
I20260807 22:50:32.553990 16092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a: Bootstrap starting.
I20260807 22:50:32.557749 16092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:32.558902 16092 log.cc:824] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:32.561969 16092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a: No bootstrap required, opened a new log
Aug 07 22:50:32 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:32.570438 16092 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } }
I20260807 22:50:32.570840 16092 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:32.570967 16092 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e2f4020a754f109889696b5202d12a, State: Initialized, Role: FOLLOWER
I20260807 22:50:32.571729 16092 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } }
I20260807 22:50:32.572028 16092 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:32.572177 16092 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:32.572359 16092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:32.573377 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:32.551887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.9.194.254:33533)
0807 22:50:32.552700 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:32.552712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:32.552752 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:50:32.554486 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:32.554506 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:32.554873 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:32.555191 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:32.557095 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:32.558517 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:32.560142 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:32.560797 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:32.560943 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:32.565639 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:32.565647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:32.565661 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:32.565698 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:32.568894 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:32.569584 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:32.569590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:32.569596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:32.569773 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:32.570168 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:32.570173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:32.570175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:32.570689 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:32.571003 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:32.571460 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 22:50:32.571907 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":392,"threads_started":1}
I20260807 22:50:32.575635 16092 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } }
I20260807 22:50:32.576236 16092 leader_election.cc:302] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e2f4020a754f109889696b5202d12a; no voters:
I20260807 22:50:32.576874 16092 leader_election.cc:288] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:32.577018 16098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:32.577731 16098 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [term 1 LEADER]: Becoming Leader. State: Replica: f7e2f4020a754f109889696b5202d12a, State: Running, Role: LEADER
I20260807 22:50:32.578212 16098 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } }
I20260807 22:50:32.578953 16092 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:32.581231 16099 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e2f4020a754f109889696b5202d12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } } }
I20260807 22:50:32.581679 16099 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:32.582415 16101 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e2f4020a754f109889696b5202d12a. Latest consensus state: current_term: 1 leader_uuid: "f7e2f4020a754f109889696b5202d12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e2f4020a754f109889696b5202d12a" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33533 } } }
I20260807 22:50:32.583026 16101 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:32.584656 16108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:32.589565 16108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:32.598327 16108 catalog_manager.cc:1347] Generated new cluster ID: 63da59710cb449b4ad6d53e17db38389
I20260807 22:50:32.598443 16108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:32.623311 16108 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:32.625131 16108 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:32.633001 16108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e2f4020a754f109889696b5202d12a: Generated new TSK 0
I20260807 22:50:32.633826 16108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:32.691177 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:33.024192 16120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:33.024523 16120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:33.024696 16120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:33.031883 16120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:33.032045 16120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:33.032229 16120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:33.041896 16120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:33.043236 16120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:33.044656 16120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:33.057866 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:33.057952 16125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:33.057952 16126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:33.059046 16120 server_base.cc:1034] running on GCE node
I20260807 22:50:33.059719 16120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:33.060626 16120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:33.061853 16120 hybrid_clock.cc:648] HybridClock initialized: now 1786143033061819 us; error 45 us; skew 500 ppm
I20260807 22:50:33.062157 16120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:33 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143033, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:33.068238 16120 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:33.069654 16120 webserver.cc:460] Webserver started at http://127.9.194.193:33219/ using document root <none> and password file <none>
I20260807 22:50:33.070343 16120 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:33.070493 16120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:33.070806 16120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:33.074347 16120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "684faaf9d1d748dda7cdb2430687e65f"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "30efdcebd6634c23d853a06c8a0bd378"
server_key_iv: "7ff168336c36e51127f93698cf3bdc24"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.075170 16120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "684faaf9d1d748dda7cdb2430687e65f"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "30efdcebd6634c23d853a06c8a0bd378"
server_key_iv: "7ff168336c36e51127f93698cf3bdc24"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.080359 16120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:50:33.084033 16136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.085381 16120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:50:33.085584 16120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "684faaf9d1d748dda7cdb2430687e65f"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "30efdcebd6634c23d853a06c8a0bd378"
server_key_iv: "7ff168336c36e51127f93698cf3bdc24"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.085807 16120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:33.124008 16120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:33.128189 16120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:33.128507 16120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:33.129515 16120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:33.130872 16120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:33.130985 16120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.131141 16120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:33.131212 16120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.165164 16120 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:44817
I20260807 22:50:33.165222 16256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:44817 every 8 connection(s)
I20260807 22:50:33.166509 16120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:33.176220 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16120
I20260807 22:50:33.176437 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:33.177094 9995 external_mini_cluster.cc:1428] Setting key 1ac5f6c1fc496609f2798a46a021f952
Aug 07 22:50:33 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143033, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:33.189157 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:33.169192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:43105 (local address 127.9.194.254:33533)
0807 22:50:33.169643 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:33.169648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:33.170626 (+ 978us) server_negotiation.cc:407] Connection header received
0807 22:50:33.172566 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:33.172574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:33.172667 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:33.172795 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:33.174250 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.174991 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.176716 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.177400 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.177537 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:33.181904 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:33.181913 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:33.181920 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:33.181959 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:33.185137 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:33.186091 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:33.186098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:33.186103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:33.186253 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:33.186834 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:33.186838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:33.186842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:33.187255 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:33.187457 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:33.188463 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 22:50:33.188726 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260807 22:50:33.190338 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:33.169737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33533 (local address 127.9.194.193:43105)
0807 22:50:33.170451 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:33.170492 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:33.172299 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:33.173023 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:33.173031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:33.173536 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:33.174082 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:33.174098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.175164 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.175169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:33.176570 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:33.176591 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.177636 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.177646 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:33.177937 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:33.178711 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:33.178733 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:33.181649 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:33.185325 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:33.185333 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:33.185348 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:33.185928 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:33.186429 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:33.186435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:33.186439 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:33.186705 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:33.187537 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:33.187546 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:33.188069 (+ 523us) client_negotiation.cc:769] Sending connection context
0807 22:50:33.188352 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 22:50:33.188924 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":288,"threads_started":1}
I20260807 22:50:33.192081 16257 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33533
I20260807 22:50:33.192425 16257 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:33.193188 16257 heartbeater.cc:511] Master 127.9.194.254:33533 requested a full tablet report, sending...
I20260807 22:50:33.195266 16055 ts_manager.cc:194] Registered new tserver with Master: 684faaf9d1d748dda7cdb2430687e65f (127.9.194.193:44817)
I20260807 22:50:33.196800 16055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:43105
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:33.229101 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:33.555815 16263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:33.556082 16263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:33.556260 16263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:33.563473 16263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:33.563629 16263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:33.563829 16263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:33.573529 16263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:33.574880 16263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:33.576275 16263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:33.589553 16268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:33.589637 16269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:33.590206 16271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:33.590780 16263 server_base.cc:1034] running on GCE node
I20260807 22:50:33.591415 16263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:33.592447 16263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:33.593691 16263 hybrid_clock.cc:648] HybridClock initialized: now 1786143033593627 us; error 57 us; skew 500 ppm
I20260807 22:50:33.594002 16263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:33 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143033, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:33.597150 16263 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:33.598621 16263 webserver.cc:460] Webserver started at http://127.9.194.194:39241/ using document root <none> and password file <none>
I20260807 22:50:33.599310 16263 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:33.599493 16263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:33.599814 16263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:33.602818 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "6275bf2d9e454dc7939b57d431329605"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "7f9207a14cfd447ec52a5ee479b84c43"
server_key_iv: "20b86c3011d36a3c23363214e8a6173f"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.603708 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "6275bf2d9e454dc7939b57d431329605"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "7f9207a14cfd447ec52a5ee479b84c43"
server_key_iv: "20b86c3011d36a3c23363214e8a6173f"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.608794 16263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:50:33.612217 16278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.613492 16263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:50:33.613667 16263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "6275bf2d9e454dc7939b57d431329605"
format_stamp: "Formatted at 2026-08-07 22:50:33 on dist-test-slave-5k54"
server_key: "7f9207a14cfd447ec52a5ee479b84c43"
server_key_iv: "20b86c3011d36a3c23363214e8a6173f"
server_key_version: "encryptionkey@0"
I20260807 22:50:33.613873 16263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:33.631726 16263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:33.635820 16263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:33.636123 16263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:33.637063 16263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:33.638428 16263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:33.638531 16263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.638692 16263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:33.638767 16263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:33.671986 16263 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:37253
I20260807 22:50:33.672032 16401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:37253 every 8 connection(s)
I20260807 22:50:33.673314 16263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:33.675136 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16263
I20260807 22:50:33.675315 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:33.675840 9995 external_mini_cluster.cc:1428] Setting key 55b82d8b66d76e54ef0074ce53926669
Aug 07 22:50:33 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143033, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:33.694576 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:33.677080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:46179 (local address 127.9.194.254:33533)
0807 22:50:33.677278 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:33.677288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:33.677812 (+ 524us) server_negotiation.cc:407] Connection header received
0807 22:50:33.679851 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:33.679856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:33.679945 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:33.680052 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:33.681529 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.682281 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.683956 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.684396 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.684578 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:33.688049 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:33.688054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:33.688057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:33.688089 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:33.691494 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:33.692224 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:33.692230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:33.692234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:33.692380 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:33.692810 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:33.692815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:33.692819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:33.693102 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:33.693264 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:33.693936 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 22:50:33.694195 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:50:33.695616 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:33.676656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33533 (local address 127.9.194.194:46179)
0807 22:50:33.677648 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:33.677691 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:33.679577 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:33.680299 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:33.680308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:33.680807 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:33.681354 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:33.681371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.682438 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.682444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:33.683813 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:33.683824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:33.684673 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:33.684681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:33.684910 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:33.685598 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:33.685626 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:33.687826 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:33.691651 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:33.691658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:33.691670 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:33.692087 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:33.692491 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:33.692497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:33.692500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:33.692686 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:33.693237 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:33.693244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:33.693570 (+ 326us) client_negotiation.cc:769] Sending connection context
0807 22:50:33.693847 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 22:50:33.694387 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":572,"threads_started":1}
I20260807 22:50:33.697242 16402 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33533
I20260807 22:50:33.697597 16402 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:33.698356 16402 heartbeater.cc:511] Master 127.9.194.254:33533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:50:33.699910 16055 ts_manager.cc:194] Registered new tserver with Master: 6275bf2d9e454dc7939b57d431329605 (127.9.194.194:37253)
I20260807 22:50:33.700919 16055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:46179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:33.729893 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:34.055539 16407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:34.055821 16407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:34.056008 16407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:34.063163 16407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:34.063267 16407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:34.063490 16407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:34.073165 16407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:34.074451 16407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:34.075860 16407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:34.089357 16414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:34.089464 16416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:34.089460 16412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:34.090464 16407 server_base.cc:1034] running on GCE node
I20260807 22:50:34.091141 16407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:34.092217 16407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:34.093405 16407 hybrid_clock.cc:648] HybridClock initialized: now 1786143034093350 us; error 51 us; skew 500 ppm
I20260807 22:50:34.093727 16407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.097075 16407 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:34.098486 16407 webserver.cc:460] Webserver started at http://127.9.194.195:36283/ using document root <none> and password file <none>
I20260807 22:50:34.099169 16407 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:34.099304 16407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:34.099725 16407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:34.102766 16407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "b1275e89e2594f609e807c836dab1812"
format_stamp: "Formatted at 2026-08-07 22:50:34 on dist-test-slave-5k54"
server_key: "a498b4404150fbe9a4b6d53fdc965bb2"
server_key_iv: "9a4f931d98090350d34a3e295ea11d65"
server_key_version: "encryptionkey@0"
I20260807 22:50:34.103695 16407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "b1275e89e2594f609e807c836dab1812"
format_stamp: "Formatted at 2026-08-07 22:50:34 on dist-test-slave-5k54"
server_key: "a498b4404150fbe9a4b6d53fdc965bb2"
server_key_iv: "9a4f931d98090350d34a3e295ea11d65"
server_key_version: "encryptionkey@0"
I20260807 22:50:34.108878 16407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 22:50:34.112423 16423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:34.113686 16407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:50:34.113867 16407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "b1275e89e2594f609e807c836dab1812"
format_stamp: "Formatted at 2026-08-07 22:50:34 on dist-test-slave-5k54"
server_key: "a498b4404150fbe9a4b6d53fdc965bb2"
server_key_iv: "9a4f931d98090350d34a3e295ea11d65"
server_key_version: "encryptionkey@0"
I20260807 22:50:34.114084 16407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:34.125031 16407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:34.129426 16407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:34.129717 16407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:34.130686 16407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:34.132128 16407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:34.132233 16407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:34.132381 16407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:34.132462 16407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:34.166181 16407 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:40597
I20260807 22:50:34.166201 16543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:40597 every 8 connection(s)
I20260807 22:50:34.167742 16407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:34.171480 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16407
I20260807 22:50:34.171684 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:34.172207 9995 external_mini_cluster.cc:1428] Setting key 8eb29e6a6b7ad1c38e9cff15f6bc7198
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:34.188388 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.170508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:57001 (local address 127.9.194.254:33533)
0807 22:50:34.170729 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.170735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.171980 (+ 1245us) server_negotiation.cc:407] Connection header received
0807 22:50:34.173947 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.173953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.174041 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.174157 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.175636 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.176374 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.177898 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.178311 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.178446 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.182055 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.182059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.182063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.182095 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.184974 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.185751 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.185756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.185759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.185943 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.186451 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.186455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.186457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.186842 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.186999 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.187796 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.188014 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:50:34.189595 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.170936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33533 (local address 127.9.194.195:57001)
0807 22:50:34.171807 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:34.171850 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:34.173651 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:34.174384 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:34.174392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:34.174880 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:34.175471 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:34.175500 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.176531 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.176544 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:34.177746 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:34.177768 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.178549 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.178556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:34.178779 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.179564 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:34.179585 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:34.181860 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:34.185093 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:34.185100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:34.185111 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:34.185593 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:34.186057 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:34.186062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:34.186065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:34.186323 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:34.186973 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:34.186981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:34.187341 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 22:50:34.187677 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 22:50:34.188188 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":372,"threads_started":1}
I20260807 22:50:34.191125 16544 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33533
I20260807 22:50:34.191490 16544 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:34.192281 16544 heartbeater.cc:511] Master 127.9.194.254:33533 requested a full tablet report, sending...
I20260807 22:50:34.193699 16055 ts_manager.cc:194] Registered new tserver with Master: b1275e89e2594f609e807c836dab1812 (127.9.194.195:40597)
I20260807 22:50:34.194494 16055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:57001
I20260807 22:50:34.199445 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:50:34.199581 16257 heartbeater.cc:503] Master 127.9.194.254:33533 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/test-admin.keytab.
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.230661 9995 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 22:50:34.235934 16551 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:34.233525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.9.194.193:44817)
0807 22:50:34.234169 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.234177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.234203 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:50:34.234278 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.234285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.234497 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.234632 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.235480 (+ 848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59394: BlockingRecv error: recv got EOF from 127.0.0.1:59394 (error 108)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":342,"threads_started":1}
W20260807 22:50:34.236035 9995 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.237628 9995 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.238983 9995 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:34.254125 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.240534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.9.194.193:44817)
0807 22:50:34.240741 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.240747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.240991 (+ 244us) server_negotiation.cc:407] Connection header received
0807 22:50:34.241198 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.241202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.241259 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.241359 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.242368 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.243312 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.244589 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.245166 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.245307 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.248714 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.248725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.248738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.248774 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.251359 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.252080 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.252088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.252090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.252213 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.252583 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.252590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.252593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.252936 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.253233 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.253572 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.253833 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 22:50:34.259634 16551 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:34.258126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.9.194.193:44817)
0807 22:50:34.258323 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.258328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.258468 (+ 140us) server_negotiation.cc:407] Connection header received
0807 22:50:34.258575 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.258579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.258630 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.258702 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.259330 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59404: BlockingRecv error: recv got EOF from 127.0.0.1:59404 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 22:50:34.259754 9995 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.261183 9995 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.262434 9995 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:34.275555 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.263662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.9.194.193:44817)
0807 22:50:34.263860 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.263866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.264065 (+ 199us) server_negotiation.cc:407] Connection header received
0807 22:50:34.264168 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.264172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.264218 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.264290 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.265123 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.265790 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.266902 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.267318 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.267404 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.270667 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.270672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.270675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.270703 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.273253 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.273818 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.273824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.273827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.273969 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.274308 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.274311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.274313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.274567 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.274748 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.275030 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.275212 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 22:50:34.279552 16551 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:34.277902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.9.194.193:44817)
0807 22:50:34.278108 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.278113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.278285 (+ 172us) server_negotiation.cc:407] Connection header received
0807 22:50:34.278376 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.278379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.278437 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.278515 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.279234 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59410: BlockingRecv error: recv got EOF from 127.0.0.1:59410 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 22:50:34.279671 9995 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142983724342-9995-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.281174 9995 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:34.282562 9995 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:34.295850 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.283975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.9.194.193:44817)
0807 22:50:34.284160 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.284166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.284365 (+ 199us) server_negotiation.cc:407] Connection header received
0807 22:50:34.284479 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.284483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.284535 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.284608 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.285516 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.286197 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.287323 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.287786 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.287876 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.291252 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.291257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.291259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.291287 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.293701 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.294230 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.294234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.294236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.294373 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.294674 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.294677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.294679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.294912 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.295157 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.295293 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.295506 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:50:34.297351 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16120
I20260807 22:50:34.307938 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.298034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.9.194.193:44817)
0807 22:50:34.298172 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.298176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.298510 (+ 334us) server_negotiation.cc:407] Connection header received
0807 22:50:34.298618 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.298621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.298669 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.298743 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.299600 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.300225 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.301328 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.301699 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.301803 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.303674 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.303678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.303681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.303708 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.305864 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.306401 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.306405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.306406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.306528 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.306819 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.306822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.306824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.307051 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.307236 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.307524 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.307674 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:50:34.308459 16252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:34.392375 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16120
I20260807 22:50:34.411258 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16263
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:34.426846 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.412192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39172 (local address 127.9.194.194:37253)
0807 22:50:34.412737 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.412744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.412769 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:50:34.412832 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.412838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.413049 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.413230 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.414235 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.415240 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.416547 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.417089 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.417232 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.421382 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.421393 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.421407 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.421442 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.424307 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.424981 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.424987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.424990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.425108 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.425443 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.425446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.425447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.425771 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.426010 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.426257 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.426514 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":244,"threads_started":1}
I20260807 22:50:34.428203 16397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:34.512760 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16263
I20260807 22:50:34.532056 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16407
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:34.548444 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:34.533153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.9.194.195:40597)
0807 22:50:34.533796 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:34.533804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:34.533838 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:50:34.533901 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:34.533907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:34.534112 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:34.534281 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:34.535368 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.536403 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.537682 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:34.538254 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:34.538407 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:34.542690 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:34.542701 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:34.542715 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:34.542750 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:34.545683 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.546372 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.546380 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.546382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.546499 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:34.546905 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:34.546910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:34.546913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:34.547252 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:34.547596 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:34.547880 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 22:50:34.548143 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":313,"threads_started":1}
I20260807 22:50:34.549742 16539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:34.631959 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16407
I20260807 22:50:34.650070 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16021
I20260807 22:50:34.651305 16086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:34.754366 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16021
2026-08-07T22:50:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4054 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16575](info): set up 2 sockets
Aug 07 22:50:34 dist-test-slave-5k54 krb5kdc[16575](info): commencing operation
krb5kdc: starting...
W20260807 22:50:35.892544 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:36 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab.
Entry for principal oryx/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab.
I20260807 22:50:36.126659 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39591
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab
--principal=oryx/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:39591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:36.471024 16591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:36.471377 16591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:36.471565 16591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:36.478673 16591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:36.478819 16591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:36.478938 16591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:36.479054 16591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:36.479142 16591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:36.489149 16591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:39591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab
--principal=oryx/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:39591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:36.490545 16591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:36.491879 16591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:36.504067 16599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:36.504402 16601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:36.504091 16597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:36.504829 16591 server_base.cc:1034] running on GCE node
I20260807 22:50:36.505592 16591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:36.507040 16591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:36.508239 16591 hybrid_clock.cc:648] HybridClock initialized: now 1786143036508179 us; error 54 us; skew 500 ppm
I20260807 22:50:36.508600 16591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:36 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:36.511793 16591 init.cc:377] Logged in from keytab as oryx/127.9.194.254@KRBTEST.COM (short username oryx)
I20260807 22:50:36.513257 16591 webserver.cc:460] Webserver started at http://127.9.194.254:37575/ using document root <none> and password file <none>
I20260807 22:50:36.513935 16591 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:36.514091 16591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:36.514391 16591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:36.517388 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "afc993b5d4de4bb08faad4f4eb8a55b1"
format_stamp: "Formatted at 2026-08-07 22:50:36 on dist-test-slave-5k54"
server_key: "0b2a76a437a3500afe926f47755bb765"
server_key_iv: "1c127019bf09c12e2fa10f78e16e3714"
server_key_version: "encryptionkey@0"
I20260807 22:50:36.518221 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "afc993b5d4de4bb08faad4f4eb8a55b1"
format_stamp: "Formatted at 2026-08-07 22:50:36 on dist-test-slave-5k54"
server_key: "0b2a76a437a3500afe926f47755bb765"
server_key_iv: "1c127019bf09c12e2fa10f78e16e3714"
server_key_version: "encryptionkey@0"
I20260807 22:50:36.523361 16591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 22:50:36.526772 16609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:36.528187 16591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:36.528380 16591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "afc993b5d4de4bb08faad4f4eb8a55b1"
format_stamp: "Formatted at 2026-08-07 22:50:36 on dist-test-slave-5k54"
server_key: "0b2a76a437a3500afe926f47755bb765"
server_key_iv: "1c127019bf09c12e2fa10f78e16e3714"
server_key_version: "encryptionkey@0"
I20260807 22:50:36.528576 16591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:36.545431 16591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:36.549942 16591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:36.550258 16591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:36.567809 16591 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:39591
I20260807 22:50:36.567821 16665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:39591 every 8 connection(s)
I20260807 22:50:36.569247 16591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:36.572943 16666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:36.579466 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16591
I20260807 22:50:36.579711 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:36.580564 9995 external_mini_cluster.cc:1428] Setting key 21005c8e1d897a20d4b8456d5f719d4f
I20260807 22:50:36.584301 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Bootstrap starting.
I20260807 22:50:36.587805 16666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:36.588963 16666 log.cc:824] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:36.592162 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: No bootstrap required, opened a new log
Aug 07 22:50:36 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:36.601604 16666 raft_consensus.cc:348] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } }
I20260807 22:50:36.602062 16666 raft_consensus.cc:374] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:36.602182 16666 raft_consensus.cc:729] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afc993b5d4de4bb08faad4f4eb8a55b1, State: Initialized, Role: FOLLOWER
I20260807 22:50:36.602921 16666 consensus_queue.cc:260] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } }
I20260807 22:50:36.603179 16666 raft_consensus.cc:388] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:36.603331 16666 raft_consensus.cc:482] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:36.603536 16666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:36.604313 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:36.582783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.9.194.254:39591)
0807 22:50:36.583655 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:36.583667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:36.583708 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:36.585685 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:36.585710 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:36.586097 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:36.586423 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:36.588417 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:36.589772 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:36.591151 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:36.592240 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:36.592446 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:36.596577 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:36.596584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:36.596599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:36.596643 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:36.599791 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:36.600622 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:36.600628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:36.600634 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:36.600804 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:36.601197 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:36.601200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:36.601202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:36.601733 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:36.601964 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:36.602637 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 22:50:36.603077 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":430,"threads_started":1}
I20260807 22:50:36.605976 16666 raft_consensus.cc:504] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } }
I20260807 22:50:36.606472 16666 leader_election.cc:302] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afc993b5d4de4bb08faad4f4eb8a55b1; no voters:
I20260807 22:50:36.607247 16666 leader_election.cc:288] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:36.607366 16671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:36.608182 16671 raft_consensus.cc:686] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 LEADER]: Becoming Leader. State: Replica: afc993b5d4de4bb08faad4f4eb8a55b1, State: Running, Role: LEADER
I20260807 22:50:36.608677 16671 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } }
I20260807 22:50:36.609421 16666 sys_catalog.cc:564] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:36.611683 16672 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } }
I20260807 22:50:36.612007 16672 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:36.612694 16673 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc993b5d4de4bb08faad4f4eb8a55b1. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } }
I20260807 22:50:36.613015 16673 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:36.615061 16676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:36.620656 16676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:36.629513 16676 catalog_manager.cc:1347] Generated new cluster ID: 607daf0a5a7b461e9a550e7203095e48
I20260807 22:50:36.629642 16676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:36.650417 16676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:36.652325 16676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:36.661724 16676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Generated new TSK 0
I20260807 22:50:36.662559 16676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
I20260807 22:50:36.730536 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:37.065312 16692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:37.065591 16692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:37.065753 16692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:37.072966 16692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:37.073114 16692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:37.073305 16692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:37.082880 16692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:37.084268 16692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:37.085618 16692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:37.098541 16697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:37.098505 16701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:37.098690 16698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:37.100194 16692 server_base.cc:1034] running on GCE node
I20260807 22:50:37.100876 16692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:37.101824 16692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:37.103027 16692 hybrid_clock.cc:648] HybridClock initialized: now 1786143037102967 us; error 55 us; skew 500 ppm
I20260807 22:50:37.103348 16692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:37 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143037, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:37.106676 16692 init.cc:377] Logged in from keytab as oryx/127.9.194.193@KRBTEST.COM (short username oryx)
I20260807 22:50:37.108220 16692 webserver.cc:460] Webserver started at http://127.9.194.193:36953/ using document root <none> and password file <none>
I20260807 22:50:37.108974 16692 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:37.109140 16692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:37.109548 16692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:37.112601 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "2da55ca5ba27443c9abc68f0d333e203"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "36fcfee442047a5276c9747ca351d10d"
server_key_iv: "b802fc8efd22fef295d4e430576c5971"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.113394 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "2da55ca5ba27443c9abc68f0d333e203"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "36fcfee442047a5276c9747ca351d10d"
server_key_iv: "b802fc8efd22fef295d4e430576c5971"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.118780 16692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:50:37.122171 16708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.123673 16692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:37.123871 16692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "2da55ca5ba27443c9abc68f0d333e203"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "36fcfee442047a5276c9747ca351d10d"
server_key_iv: "b802fc8efd22fef295d4e430576c5971"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.124087 16692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:37.136631 16692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:37.140467 16692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:37.140771 16692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:37.141696 16692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:37.142979 16692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:37.143055 16692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.143217 16692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:37.143278 16692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.176580 16692 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:43437
I20260807 22:50:37.176641 16831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:43437 every 8 connection(s)
I20260807 22:50:37.177865 16692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:37.185499 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16692
I20260807 22:50:37.185735 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:37.186475 9995 external_mini_cluster.cc:1428] Setting key 1cd6d4ce682e50785ce35e56897bfb27
Aug 07 22:50:37 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143037, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:37.201104 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:37.180641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:51533 (local address 127.9.194.254:39591)
0807 22:50:37.181112 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:37.181119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:37.182115 (+ 996us) server_negotiation.cc:407] Connection header received
0807 22:50:37.184000 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:37.184005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:37.184073 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:37.184167 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:37.186045 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.186875 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.188922 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.189325 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.189416 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:37.193601 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:37.193610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:37.193617 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:37.193662 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:37.197274 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:37.198109 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:37.198114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:37.198116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:37.198237 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:37.198736 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:37.198739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:37.198741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:37.199142 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:37.199298 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:37.200334 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0807 22:50:37.200602 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260807 22:50:37.202170 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:37.181237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:51533)
0807 22:50:37.181946 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:37.181990 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:37.183736 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:37.184373 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:37.184382 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:37.185006 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:37.185854 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:37.185880 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.187048 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.187055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:37.188751 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:37.188767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.189544 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.189553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:37.189908 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:37.190638 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:37.190672 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:37.193378 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:37.197406 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:37.197413 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:37.197425 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:37.197967 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:37.198343 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:37.198350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:37.198356 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:37.198613 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:37.199354 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:37.199362 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:37.199868 (+ 506us) client_negotiation.cc:769] Sending connection context
0807 22:50:37.200187 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 22:50:37.200754 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":270,"threads_started":1}
I20260807 22:50:37.203799 16833 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:37.204128 16833 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:37.204892 16833 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:37.206986 16629 ts_manager.cc:194] Registered new tserver with Master: 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:37.208750 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:51533
WARNING: no policy specified for oryx/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
I20260807 22:50:37.242654 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:37.576315 16840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:37.576681 16840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:37.576858 16840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:37.589112 16840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:37.589283 16840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:37.589480 16840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:37.600086 16840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:37.601361 16840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:37.602670 16840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:37.615907 16845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:37.615932 16846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:37.615912 16848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:37.616786 16840 server_base.cc:1034] running on GCE node
I20260807 22:50:37.617393 16840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:37.618255 16840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:37.619520 16840 hybrid_clock.cc:648] HybridClock initialized: now 1786143037619470 us; error 57 us; skew 500 ppm
I20260807 22:50:37.619866 16840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:37 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143037, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:37.623220 16840 init.cc:377] Logged in from keytab as oryx/127.9.194.194@KRBTEST.COM (short username oryx)
I20260807 22:50:37.624730 16840 webserver.cc:460] Webserver started at http://127.9.194.194:38217/ using document root <none> and password file <none>
I20260807 22:50:37.625453 16840 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:37.625597 16840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:37.625944 16840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:37.628949 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "1a18b33621054bb0b3e29e5f4b63b524"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "7244380d6434b8afdca1acd1897ac463"
server_key_iv: "31abc4fb79b42f08f60b0067f3b0a8b7"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.629714 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "1a18b33621054bb0b3e29e5f4b63b524"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "7244380d6434b8afdca1acd1897ac463"
server_key_iv: "31abc4fb79b42f08f60b0067f3b0a8b7"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.635082 16840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:50:37.638453 16855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.639847 16840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:37.640021 16840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "1a18b33621054bb0b3e29e5f4b63b524"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "7244380d6434b8afdca1acd1897ac463"
server_key_iv: "31abc4fb79b42f08f60b0067f3b0a8b7"
server_key_version: "encryptionkey@0"
I20260807 22:50:37.640208 16840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:37.657342 16840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:37.662062 16840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:37.662382 16840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:37.663368 16840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:37.664731 16840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:37.664855 16840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.665007 16840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:37.665081 16840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:37.699198 16840 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:39959
I20260807 22:50:37.699246 16979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:39959 every 8 connection(s)
I20260807 22:50:37.700598 16840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:37.705264 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16840
I20260807 22:50:37.705472 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:37.705936 9995 external_mini_cluster.cc:1428] Setting key 586e12274e1e9285f68b86fba350ee49
Aug 07 22:50:37 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143037, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:37.723696 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:37.703176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:58317 (local address 127.9.194.254:39591)
0807 22:50:37.703683 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:37.703688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:37.704593 (+ 905us) server_negotiation.cc:407] Connection header received
0807 22:50:37.706676 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:37.706684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:37.706769 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:37.706896 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:37.708772 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.710081 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.711827 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.712571 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.712717 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:37.716724 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:37.716729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:37.716732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:37.716765 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:37.720450 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:37.721137 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:37.721145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:37.721151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:37.721313 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:37.721712 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:37.721719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:37.721725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:37.722103 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:37.722258 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:37.722902 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 22:50:37.723159 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":292,"threads_started":1}
I20260807 22:50:37.724314 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:37.703731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:58317)
0807 22:50:37.704420 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:37.704466 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:37.706337 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:37.707193 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:37.707204 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:37.708006 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:37.708594 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:37.708612 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.710237 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.710241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:37.711687 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:37.711700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:37.712776 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:37.712780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:37.712982 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:37.713680 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:37.713707 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:37.716467 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:37.720580 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:37.720586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:37.720597 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:37.720996 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:37.721418 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:37.721421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:37.721423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:37.721599 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:37.722219 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:37.722224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:37.722530 (+ 306us) client_negotiation.cc:769] Sending connection context
0807 22:50:37.722742 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 22:50:37.723136 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":262,"threads_started":1}
I20260807 22:50:37.725839 16980 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:37.726181 16980 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:37.727100 16980 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:37.728835 16629 ts_manager.cc:194] Registered new tserver with Master: 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959)
I20260807 22:50:37.729662 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:58317
WARNING: no policy specified for oryx/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
I20260807 22:50:37.763124 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:38.094473 16986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:38.094823 16986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:38.095010 16986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:38.102216 16986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:38.102373 16986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:38.102564 16986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:38.112620 16986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:39591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:38.113972 16986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:38.115851 16986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:38.130769 16991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:38.130728 16994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:38.130822 16992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:38.131705 16986 server_base.cc:1034] running on GCE node
I20260807 22:50:38.132398 16986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:38.133384 16986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:38.134601 16986 hybrid_clock.cc:648] HybridClock initialized: now 1786143038134537 us; error 54 us; skew 500 ppm
I20260807 22:50:38.134928 16986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143038, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:38.141808 16986 init.cc:377] Logged in from keytab as oryx/127.9.194.195@KRBTEST.COM (short username oryx)
I20260807 22:50:38.143257 16986 webserver.cc:460] Webserver started at http://127.9.194.195:33863/ using document root <none> and password file <none>
I20260807 22:50:38.144043 16986 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:38.144186 16986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:38.144516 16986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:38.147400 16986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "517a04892fdf40c0a9a582ba9c273cd8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "a13accac299dbad704c22a1f12345785"
server_key_iv: "b1c15737362f056b438807115f954b8b"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.148254 16986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "517a04892fdf40c0a9a582ba9c273cd8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "a13accac299dbad704c22a1f12345785"
server_key_iv: "b1c15737362f056b438807115f954b8b"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.153280 16986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:38.156735 17001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:38.158044 16986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:50:38.158226 16986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "517a04892fdf40c0a9a582ba9c273cd8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "a13accac299dbad704c22a1f12345785"
server_key_iv: "b1c15737362f056b438807115f954b8b"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.158428 16986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:38.178691 16986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:38.183653 16986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:38.183893 16986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:38.184753 16986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:38.186038 16986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:38.186115 16986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:38.186206 16986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:38.186264 16986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:38.211783 16833 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
I20260807 22:50:38.225215 16986 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:43887
I20260807 22:50:38.225270 17123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:43887 every 8 connection(s)
I20260807 22:50:38.226581 16986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:38.236342 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 16986
I20260807 22:50:38.236538 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:38.237085 9995 external_mini_cluster.cc:1428] Setting key 8b10e68603b790fd2ee80035381e7daf
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143038, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:38.246795 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.229338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:38119 (local address 127.9.194.254:39591)
0807 22:50:38.229784 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.229789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.230839 (+ 1050us) server_negotiation.cc:407] Connection header received
0807 22:50:38.232709 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.232714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.232781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.232964 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.234399 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.235178 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.236855 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.237481 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.237626 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.241171 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.241176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.241180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.241218 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.243821 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.244514 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.244521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.244529 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.244689 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.245114 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.245119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.245122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.245374 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.245534 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.246205 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.246421 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260807 22:50:38.247749 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.229965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:38119)
0807 22:50:38.230673 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.230719 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.232455 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.233159 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.233167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.233673 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.234249 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.234265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.235300 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.235303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.236717 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.236728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.237726 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.237735 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.238007 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.238698 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.238720 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.240974 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.243951 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.243959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.243975 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.244389 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.244805 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.244811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.244815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.244996 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.245509 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.245517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.245837 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.246094 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.246606 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":267,"threads_started":1}
I20260807 22:50:38.249408 17124 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:38.251982 17124 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:38.253036 17124 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:38.254915 16629 ts_manager.cc:194] Registered new tserver with Master: 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:38.255820 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:38119
I20260807 22:50:38.266858 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.194.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.253.keytab.
Entry for principal oryx/127.9.194.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.253.keytab.
I20260807 22:50:38.319967 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:37461
--webserver_interface=127.9.194.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.253.keytab
--principal=oryx/127.9.194.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:39591,127.9.194.253:37461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:38.659932 17130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:38.660229 17130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:38.660386 17130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:38.667693 17130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:38.667836 17130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:38.667949 17130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:38.668051 17130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:38.668120 17130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:38.677772 17130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:39591,127.9.194.253:37461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.253.keytab
--principal=oryx/127.9.194.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:37461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.194.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
W20260807 22:50:38.678171 17130 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.194.254:39591,127.9.194.253:37461'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 22:50:38.679280 17130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:38.680641 17130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:38.692806 17135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:38.693023 17138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:38.692847 17136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:38.696847 17130 server_base.cc:1034] running on GCE node
I20260807 22:50:38.697626 17130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:38.699198 17130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:38.700434 17130 hybrid_clock.cc:648] HybridClock initialized: now 1786143038700352 us; error 63 us; skew 500 ppm
I20260807 22:50:38.700768 17130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143038, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:38.704190 17130 init.cc:377] Logged in from keytab as oryx/127.9.194.253@KRBTEST.COM (short username oryx)
I20260807 22:50:38.705747 17130 webserver.cc:460] Webserver started at http://127.9.194.253:43761/ using document root <none> and password file <none>
I20260807 22:50:38.706429 17130 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:38.706518 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:38.706784 17130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:38.709836 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/instance:
uuid: "72809db13eaa4248addf9720111c98c8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "c805802cb33b55a0a3f64855ce0e8ef9"
server_key_iv: "3cd41022f2cf74416e1bfef4fc1b3c41"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.710632 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal/instance:
uuid: "72809db13eaa4248addf9720111c98c8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "c805802cb33b55a0a3f64855ce0e8ef9"
server_key_iv: "3cd41022f2cf74416e1bfef4fc1b3c41"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.715898 17130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 22:50:38.719192 17145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:38.720531 17130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:38.720669 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "72809db13eaa4248addf9720111c98c8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "c805802cb33b55a0a3f64855ce0e8ef9"
server_key_iv: "3cd41022f2cf74416e1bfef4fc1b3c41"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.720839 17130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:38.732949 16980 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
I20260807 22:50:38.733476 17130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:38.737951 17130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:38.738234 17130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:38.756882 17130 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.253:37461
I20260807 22:50:38.756968 17199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.253:37461 every 8 connection(s)
I20260807 22:50:38.758234 17130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
I20260807 22:50:38.761951 17200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:38.763784 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 17130
I20260807 22:50:38.771796 17200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143038, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.253@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:38.786950 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.766035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.9.194.253:37461)
0807 22:50:38.766918 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.766930 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.766969 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:50:38.768642 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.768662 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.769048 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.769366 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.771149 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.772538 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.773879 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.774865 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.775069 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.779747 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.779754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.779764 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.779801 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.782997 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.783746 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.783752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.783758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.783901 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.784350 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.784357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.784365 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.784849 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.785158 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.785606 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.785994 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":444,"threads_started":1}
I20260807 22:50:38.789199 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.774265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.9.194.254:39591)
0807 22:50:38.774741 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.774747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.775339 (+ 592us) server_negotiation.cc:407] Connection header received
0807 22:50:38.775553 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.775561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.775665 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.775750 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.776860 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.778049 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.779419 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.779838 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.779980 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.783570 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.783577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.783582 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.783621 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.786463 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.787205 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.787214 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.787217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.787340 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.787729 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.787736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.787739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.788037 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.788210 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.788642 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.788857 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":232,"threads_started":1}
I20260807 22:50:38.789299 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.774307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:34976)
0807 22:50:38.775183 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.775227 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.775384 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.775842 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.775850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.776446 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.776694 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.776702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.778199 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.778202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.779275 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.779287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.780044 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.780050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.780232 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.780852 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.780882 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.783361 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.786595 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.786607 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.786613 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.787100 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.787472 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.787477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.787480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.787620 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.788172 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.788179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.788347 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.788536 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.788833 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":482,"threads_started":1}
I20260807 22:50:38.790958 17200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143038, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.253@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
I20260807 22:50:38.802277 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.791678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57948)
0807 22:50:38.791905 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.791922 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.792030 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.792307 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.792311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.792648 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.792761 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.792768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.793711 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.793716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.794537 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.794547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.795192 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.795197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.795459 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.795928 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.795944 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.797776 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.800387 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.800390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.800393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.800761 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.801065 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.801068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.801070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.801189 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.801611 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.801614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.801729 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.801834 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.802044 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":3}
I20260807 22:50:38.802476 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.791808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.9.194.253:37461)
0807 22:50:38.791965 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.791969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.791985 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:38.792129 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.792134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.792189 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.792293 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.792882 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.793551 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.794669 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.795052 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.795182 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.797965 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.797969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.797972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.797999 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.800269 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.800858 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.800862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.800864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.800970 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.801274 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.801277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.801279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.801519 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.801618 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.801920 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.802136 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":48}
I20260807 22:50:38.806934 17200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Bootstrap starting.
I20260807 22:50:38.810031 17200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:38.811220 17200 log.cc:824] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:38.814049 17200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: No bootstrap required, opened a new log
I20260807 22:50:38.821170 17200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } }
I20260807 22:50:38.821485 17200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:38.821565 17200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72809db13eaa4248addf9720111c98c8, State: Initialized, Role: FOLLOWER
I20260807 22:50:38.822080 17200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } }
I20260807 22:50:38.822873 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } } }
I20260807 22:50:38.823230 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:38.823926 17200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:50:38.833531 17219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:50:38.833673 17219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:50:38.834194 17207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:38.834519 17207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } }
I20260807 22:50:38.835987 17207 leader_election.cc:288] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591)
I20260807 22:50:38.836789 17130 master_runner.cc:428] Detected that this master 72809db13eaa4248addf9720111c98c8 is joining an existing cluster
I20260807 22:50:38.836902 17130 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.194.253:37461
I20260807 22:50:38.836750 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72809db13eaa4248addf9720111c98c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" is_pre_election: true
I20260807 22:50:38.837939 17149 leader_election.cc:302] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 72809db13eaa4248addf9720111c98c8; no voters: afc993b5d4de4bb08faad4f4eb8a55b1
I20260807 22:50:38.838737 17207 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 22:50:38.859987 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.841887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34990 (local address 127.9.194.254:39591)
0807 22:50:38.842149 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.842155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.842250 (+ 95us) server_negotiation.cc:407] Connection header received
0807 22:50:38.842512 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.842517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.842611 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.842687 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.844559 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.845734 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.847865 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.848578 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.848704 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.851475 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.851484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.851490 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.851536 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.856675 (+ 5139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.857479 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.857485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.857490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.857645 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.858147 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.858152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.858156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.858518 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.858638 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.859349 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.859612 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260807 22:50:38.861675 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.841083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:34990)
0807 22:50:38.842158 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.842181 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.842353 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.842919 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.842924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.843839 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.844379 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.844397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.845931 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.845938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.847700 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.847713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.849010 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.849018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.849236 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.849954 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.849982 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.851240 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.856826 (+ 5586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.856834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.856841 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.857334 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.857788 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.857792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.857798 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.858013 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.858756 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.858762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.859010 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.860549 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.860907 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":10517,"threads_started":1}
I20260807 22:50:38.866459 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.847616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.9.194.253:37461)
0807 22:50:38.847831 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.847837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.852043 (+ 4206us) server_negotiation.cc:407] Connection header received
0807 22:50:38.852220 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.852227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.852293 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.852358 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.853752 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.854892 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.856905 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.857647 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.857744 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.860209 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.860215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.860219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.860247 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.863571 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.864318 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.864323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.864326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.864504 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.864918 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.864923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.864926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.865275 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.865425 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.865883 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.866095 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":57}
I20260807 22:50:38.867777 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.841529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57954)
0807 22:50:38.851930 (+ 10401us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.851947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.852095 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.852563 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.852570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.853272 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.853585 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.853601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.855081 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.855088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.856766 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.856782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.858083 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.858091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.858273 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.858887 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.858915 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.860026 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.863709 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.863713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.863717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.864202 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.864633 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.864637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.864640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.864812 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.865486 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.865505 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.865648 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.866930 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.867222 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10268,"thread_start_us":6310,"threads_started":1}
WARNING: no policy specified for oryx/127.9.194.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.252@KRBTEST.COM" created.
Aug 07 22:50:38 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.254@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
I20260807 22:50:38.892839 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.870103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57962)
0807 22:50:38.873086 (+ 2983us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:38.873160 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:38.873444 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:38.874430 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:38.874439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:38.875892 (+ 1453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:38.876342 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.876356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.878578 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.878583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.880170 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:38.880188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.880996 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.881003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:38.881219 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.881845 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:38.881862 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:38.884021 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:38.889399 (+ 5378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.889413 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.889421 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.890068 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.890530 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:38.890535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:38.890539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:38.890746 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:38.891568 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:38.891580 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:38.891790 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 22:50:38.892033 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 22:50:38.892373 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2625,"thread_start_us":2426,"threads_started":1}
I20260807 22:50:38.892931 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:38.870478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.9.194.253:37461)
0807 22:50:38.872217 (+ 1739us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:38.872224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:38.873270 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 22:50:38.873744 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:38.873750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:38.873920 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:38.874089 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:38.876622 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.878310 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.880324 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:38.880820 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:38.880959 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:38.884222 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:38.884228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:38.884231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:38.884261 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:38.889237 (+ 4976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.890220 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.890229 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.890235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.890403 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:38.890895 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:38.890902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:38.890908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:38.891350 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:38.891556 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:38.892201 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 22:50:38.892467 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1440}
I20260807 22:50:38.895678 16629 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.194.253:37461: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "72809db13eaa4248addf9720111c98c8"
member_type: NON_VOTER
last_known_addr {
host: "127.9.194.253"
port: 37461
}
attrs {
promote: true
}
}
dest_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1"
cas_config_opid_index: -1
I20260807 22:50:38.897377 16629 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } }
I20260807 22:50:38.901898 17173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:38.901433 17232 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 72809db13eaa4248addf9720111c98c8 (127.9.194.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.905598 17173 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.252.keytab.
Entry for principal oryx/127.9.194.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.252.keytab.
I20260807 22:50:38.907230 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:34959
--webserver_interface=127.9.194.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.252.keytab
--principal=oryx/127.9.194.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:50:38.906143 17232 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.194.253:37461
I20260807 22:50:38.908425 17232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:38.909011 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.909565 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:38.922693 17130 master.cc:537] Master@127.9.194.253:37461 shutting down...
I20260807 22:50:38.949637 17173 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 72809db13eaa4248addf9720111c98c8 (127.9.194.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.967991 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc993b5d4de4bb08faad4f4eb8a55b1. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.968263 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:38.968694 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.968883 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:38.968415 17238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:38.969201 17238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:38.972254 17130 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Raft consensus shutting down.
I20260807 22:50:38.972734 17130 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Raft consensus is shut down!
I20260807 22:50:38.972903 17130 tablet_replica.cc:330] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: stopping tablet replica
I20260807 22:50:38.982719 17130 master.cc:552] Master@127.9.194.253:37461 shutdown complete.
I20260807 22:50:38.982857 17130 master_runner.cc:305] Clearing existing system tablet
I20260807 22:50:38.983362 17130 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:38.983492 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:38.986527 17130 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:50:38.988266 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:38.988626 17130 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:38.988894 17130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:50:38.988981 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "72809db13eaa4248addf9720111c98c8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "c805802cb33b55a0a3f64855ce0e8ef9"
server_key_iv: "3cd41022f2cf74416e1bfef4fc1b3c41"
server_key_version: "encryptionkey@0"
I20260807 22:50:38.993917 17130 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:39.000872 17130 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:50:39.001055 17130 log.cc:1196] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 22:50:39.001543 17130 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Deleting consensus metadata
I20260807 22:50:39.001868 17130 master_runner.cc:315] Copying system tablet from 127.9.194.254:39591
I20260807 22:50:39.004805 17130 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: tablet copy: Beginning tablet copy session from remote peer at address 127.9.194.254:39591
I20260807 22:50:39.021677 17252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.005751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35004)
0807 22:50:39.006515 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.006540 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.006771 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.007415 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.007446 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.007951 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.008245 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.008258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.009955 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.009961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.011536 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.011558 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.012548 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.012555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.012731 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.013448 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.013479 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.014585 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.018654 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.018660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.018665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.019217 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.019675 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.019680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.019686 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.019853 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.020493 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.020499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.020695 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.020868 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.021188 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":444,"threads_started":1}
I20260807 22:50:39.021700 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.005828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.9.194.254:39591)
0807 22:50:39.006025 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.006032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.006642 (+ 610us) server_negotiation.cc:407] Connection header received
0807 22:50:39.007018 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.007028 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.007196 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.007386 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.008436 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.009735 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.011676 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.012382 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.012510 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.014800 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.014807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.014811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.014850 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.018501 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.019348 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.019354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.019357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.019559 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.019972 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.019977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.019980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.020353 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.020533 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.021028 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.021300 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:39.022879 16650 tablet_copy_service.cc:140] P afc993b5d4de4bb08faad4f4eb8a55b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 72809db13eaa4248addf9720111c98c8 ({username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:35004)
I20260807 22:50:39.023248 16650 tablet_copy_service.cc:161] P afc993b5d4de4bb08faad4f4eb8a55b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 72809db13eaa4248addf9720111c98c8 at {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:35004: session id = 72809db13eaa4248addf9720111c98c8-00000000000000000000000000000000
I20260807 22:50:39.028498 16650 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:50:39.031052 17130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:50:39.038553 17130 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: tablet copy: Starting download of 0 data blocks...
I20260807 22:50:39.038936 17130 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: tablet copy: Starting download of 1 WAL segments...
I20260807 22:50:39.042438 17130 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:50:39.047168 16650 tablet_copy_service.cc:342] P afc993b5d4de4bb08faad4f4eb8a55b1: Request end of tablet copy session 72809db13eaa4248addf9720111c98c8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:35004
I20260807 22:50:39.047447 16650 tablet_copy_service.cc:433] P afc993b5d4de4bb08faad4f4eb8a55b1: ending tablet copy session 72809db13eaa4248addf9720111c98c8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 72809db13eaa4248addf9720111c98c8
I20260807 22:50:39.060177 17130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:39.063047 17253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.068305 17255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.075552 17257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:39.076462 17130 server_base.cc:1034] running on GCE node
I20260807 22:50:39.076822 17130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:39.077478 17130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:39.078701 17130 hybrid_clock.cc:648] HybridClock initialized: now 1786143039078657 us; error 62 us; skew 500 ppm
I20260807 22:50:39.078933 17130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:39.081255 17130 init.cc:377] Logged in from keytab as oryx/127.9.194.253@KRBTEST.COM (short username oryx)
I20260807 22:50:39.082531 17130 webserver.cc:460] Webserver started at http://127.9.194.253:34133/ using document root <none> and password file <none>
I20260807 22:50:39.082890 17130 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:39.082975 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:39.086318 17130 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:50:39.088506 17265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:39.088973 17130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:50:39.089144 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "72809db13eaa4248addf9720111c98c8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "c805802cb33b55a0a3f64855ce0e8ef9"
server_key_iv: "3cd41022f2cf74416e1bfef4fc1b3c41"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.089322 17130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:39.108423 17130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:39.113327 17130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:39.147766 17130 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.253:37461
I20260807 22:50:39.147791 17320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.253:37461 every 8 connection(s)
I20260807 22:50:39.148869 17130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
I20260807 22:50:39.154407 17321 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:50:39.158864 17321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Bootstrap starting.
I20260807 22:50:39.174475 17321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:50:39.175062 17321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Bootstrap complete.
I20260807 22:50:39.176136 17321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } }
I20260807 22:50:39.176385 17321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72809db13eaa4248addf9720111c98c8, State: Initialized, Role: LEARNER
I20260807 22:50:39.176769 17321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } }
I20260807 22:50:39.181255 17324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:39.181612 17324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:39.181636 17321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:39.191323 17336 catalog_manager.cc:1259] Loaded cluster ID: 607daf0a5a7b461e9a550e7203095e48
I20260807 22:50:39.191922 17336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: loading cluster ID for follower catalog manager: success
I20260807 22:50:39.194368 17336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: acquiring CA information for follower catalog manager: success
I20260807 22:50:39.196471 17336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:50:39.258694 17124 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
W20260807 22:50:39.363353 17235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:39.363768 17235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:39.363924 17235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:39.375672 17235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:39.375819 17235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:39.375895 17235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:39.375973 17235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:39.376036 17235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:39.388041 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:39.388356 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:39.390234 17235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.252.keytab
--principal=oryx/127.9.194.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.194.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:39.391772 17235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:39.393112 17235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:50:39.394712 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.388274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57976)
0807 22:50:39.388620 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.388648 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.388825 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.389447 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.389453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.389487 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:39.389719 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.389728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.391113 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.391118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.392660 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.392672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.393851 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.393857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.394099 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.394158 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.394268 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.394405 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260807 22:50:39.394783 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.388467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.9.194.253:37461)
0807 22:50:39.389099 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.389103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.389119 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:39.389179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.389182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.389266 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.389398 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:39.389888 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.390966 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.392810 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.393685 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.393826 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.394081 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.394304 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.394435 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":330,"threads_started":1}
I20260807 22:50:39.395732 17324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc993b5d4de4bb08faad4f4eb8a55b1. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: NON_VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: true } } }
I20260807 22:50:39.396030 17324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:39.396656 17231 raft_consensus.cc:1050] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: attempting to promote NON_VOTER 72809db13eaa4248addf9720111c98c8 to VOTER
I20260807 22:50:39.397482 17231 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } }
I20260807 22:50:39.399322 17295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 LEARNER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:50:39.399961 17232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 22:50:39.403342 17231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72809db13eaa4248addf9720111c98c8 (127.9.194.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } }
I20260807 22:50:39.407028 17295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72809db13eaa4248addf9720111c98c8 (127.9.194.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } }
I20260807 22:50:39.407878 17232 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } }
I20260807 22:50:39.408138 17232 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:39.408001 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } }
I20260807 22:50:39.408308 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:39.408834 17235 server_base.cc:1034] running on GCE node
W20260807 22:50:39.408903 17342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.408864 17343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.409351 17345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:39.410298 17235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:39.410761 17324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } }
I20260807 22:50:39.411083 17324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:39.412075 17235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:39.413275 17235 hybrid_clock.cc:648] HybridClock initialized: now 1786143039413211 us; error 56 us; skew 500 ppm
I20260807 22:50:39.413691 17235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:39.417025 17235 init.cc:377] Logged in from keytab as oryx/127.9.194.252@KRBTEST.COM (short username oryx)
I20260807 22:50:39.418457 17235 webserver.cc:460] Webserver started at http://127.9.194.252:46813/ using document root <none> and password file <none>
I20260807 22:50:39.419101 17235 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:39.419201 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:39.419569 17235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:39.422572 17235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/instance:
uuid: "0ca4789937cb494bb1782b208b1bc7e4"
format_stamp: "Formatted at 2026-08-07 22:50:39 on dist-test-slave-5k54"
server_key: "0b0850888fce30010a045e9c44ef8fa4"
server_key_iv: "76c5f294443855729a5978432ea8ff7b"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.423295 17235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal/instance:
uuid: "0ca4789937cb494bb1782b208b1bc7e4"
format_stamp: "Formatted at 2026-08-07 22:50:39 on dist-test-slave-5k54"
server_key: "0b0850888fce30010a045e9c44ef8fa4"
server_key_iv: "76c5f294443855729a5978432ea8ff7b"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.428714 17235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:50:39.432467 17354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:39.433904 17235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:50:39.434064 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "0ca4789937cb494bb1782b208b1bc7e4"
format_stamp: "Formatted at 2026-08-07 22:50:39 on dist-test-slave-5k54"
server_key: "0b0850888fce30010a045e9c44ef8fa4"
server_key_iv: "76c5f294443855729a5978432ea8ff7b"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.434237 17235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:39.455669 17235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:39.459931 17235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:39.460171 17235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:39.479513 17235 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.252:34959
I20260807 22:50:39.479511 17411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.252:34959 every 8 connection(s)
I20260807 22:50:39.480957 17235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
I20260807 22:50:39.481665 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 17235
I20260807 22:50:39.486114 17412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:39.494164 17412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.252@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:39.508471 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.484036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.9.194.252:34959)
0807 22:50:39.484939 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.484955 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.485006 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:50:39.487387 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.487415 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.487963 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.488412 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.490311 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.491922 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.493177 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.493828 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.494018 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.498801 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.498817 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.498832 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.498883 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.502995 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.504689 (+ 1694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.504697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.504705 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.504956 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.505335 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.505340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.505343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.506076 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.506311 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.506946 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.507383 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":368,"threads_started":1}
I20260807 22:50:39.514021 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.497017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.9.194.254:39591)
0807 22:50:39.497233 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.497241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.498005 (+ 764us) server_negotiation.cc:407] Connection header received
0807 22:50:39.498207 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.498219 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.498285 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.498388 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.500042 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.500855 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.503155 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.503725 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.503831 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.507895 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.507900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.507904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.507945 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.510966 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.511851 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.511858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.511861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.512015 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.512407 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.512412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.512417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.512736 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.512903 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.513435 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.513670 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:50:39.514074 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.496899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35018)
0807 22:50:39.497840 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.497887 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.498078 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.498455 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.498462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.499480 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.499874 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.499893 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.501066 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.501074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.502962 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.502986 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.503916 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.503920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.504048 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.504653 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.504686 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.507650 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.511106 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.511119 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.511130 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.511729 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.512132 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.512137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.512141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.512283 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.512873 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.512880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.513070 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.513321 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.513661 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":441,"threads_started":1}
I20260807 22:50:39.515403 17412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.252@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
I20260807 22:50:39.528849 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.516202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57988)
0807 22:50:39.516409 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.516429 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.516541 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.516857 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.516861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.517247 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.517376 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.517384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.518683 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.518692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.519730 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.519742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.520483 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.520489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.520673 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.521251 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.521268 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.523195 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.526346 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.526351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.526355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.526752 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.527154 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.527159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.527162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.527301 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.527862 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.527867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.528011 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.528150 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.528501 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:50:39.528855 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.516261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.9.194.253:37461)
0807 22:50:39.516451 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.516460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.516479 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:39.516670 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.516675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.516736 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.516838 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.517519 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.518535 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.519884 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.520309 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.520448 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.523384 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.523389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.523392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.523469 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.526206 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.526876 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.526882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.526886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.527038 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.527410 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.527415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.527418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.527744 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.527900 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.528240 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.528499 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:50:39.530006 17412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.252@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
I20260807 22:50:39.542050 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.530679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.0.0.1:41812)
0807 22:50:39.530909 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.530925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.531004 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.531265 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.531269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.531685 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.531876 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.531885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.532888 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.532893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.533704 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.533714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.534410 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.534414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.534556 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.535066 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.535081 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.537091 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.540030 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.540033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.540036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.540367 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.540678 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.540681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.540683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.540830 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.541303 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.541306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.541438 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.541543 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.541722 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 22:50:39.542131 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.530887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.9.194.252:34959)
0807 22:50:39.531039 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.531043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.531058 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:50:39.531096 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.531100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.531152 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.531235 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.532039 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.532740 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.533822 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.534269 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.534366 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.537238 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.537242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.537245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.537272 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.539901 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.540469 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.540472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.540474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.540588 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.540948 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.540952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.540954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.541206 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.541314 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.541626 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.541846 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":53}
I20260807 22:50:39.547045 17412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Bootstrap starting.
I20260807 22:50:39.550160 17412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:39.551144 17412 log.cc:824] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:39.553690 17412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: No bootstrap required, opened a new log
I20260807 22:50:39.560720 17412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } }
I20260807 22:50:39.561017 17412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:39.561105 17412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca4789937cb494bb1782b208b1bc7e4, State: Initialized, Role: FOLLOWER
I20260807 22:50:39.561714 17412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } }
I20260807 22:50:39.562831 17418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } } }
I20260807 22:50:39.563232 17418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:39.563975 17412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:50:39.573413 17430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:50:39.573553 17430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:50:39.577211 17235 master_runner.cc:428] Detected that this master 0ca4789937cb494bb1782b208b1bc7e4 is joining an existing cluster
I20260807 22:50:39.577346 17235 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.194.252:34959
I20260807 22:50:39.608420 17436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.581496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:57990)
0807 22:50:39.588780 (+ 7284us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.588802 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.589367 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.589986 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.590051 (+ 65us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.590843 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.591379 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.591392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.592936 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.592941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.594674 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.594689 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.595918 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.595925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.596952 (+ 1027us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.597750 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.597780 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.599090 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.604950 (+ 5860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.604956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.604965 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.605683 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.606158 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.606165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.606170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.606353 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.607089 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.607097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.607352 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.607575 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.607886 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7125,"mutex_wait_us":1,"thread_start_us":7419,"threads_started":1}
I20260807 22:50:39.609402 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.581985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.9.194.253:37461)
0807 22:50:39.582169 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.582174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.588895 (+ 6721us) server_negotiation.cc:407] Connection header received
0807 22:50:39.589559 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.589568 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.589656 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.589731 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.591596 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.592722 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.594846 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.595518 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.595614 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.600368 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.600375 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.600380 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.600421 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.604770 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.605834 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.605840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.605845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.606017 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.606489 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.606494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.606497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.606872 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.607030 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.608822 (+ 1792us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.609059 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:50:39.611099 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.582624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.9.194.252:34959)
0807 22:50:39.582886 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.582895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.591145 (+ 8250us) server_negotiation.cc:407] Connection header received
0807 22:50:39.593834 (+ 2689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.593842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.593947 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.594027 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.595807 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.597072 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.599137 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.599897 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.600029 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.603047 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.603052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.603057 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.603091 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.607008 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.607763 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.607770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.607772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.607898 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.608297 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.608301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.608304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.609020 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.609163 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.610023 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.610278 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":84}
I20260807 22:50:39.611852 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.583266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.0.0.1:41818)
0807 22:50:39.591012 (+ 7746us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.591036 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.593656 (+ 2620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.594267 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.594274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.595169 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.595620 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.595637 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.597263 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.597268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.598997 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.599012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.600368 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.600376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.600656 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.601545 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.601572 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.602828 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.607136 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.607142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.607145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.607661 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.608031 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.608040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.608045 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.608190 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.609227 (+ 1037us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.609232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.609443 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.610223 (+ 780us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.610548 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7503,"mutex_wait_us":2,"thread_start_us":7322,"threads_started":1}
I20260807 22:50:39.613453 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.589362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35020)
0807 22:50:39.596582 (+ 7220us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.596596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.596737 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.597269 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.597274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.598163 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.598583 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.598594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.600367 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.600372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.602278 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.602292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.603561 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.603568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.603766 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.604295 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.604317 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.605705 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.609723 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.609728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.609733 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.610738 (+ 1005us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.611267 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.611272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.611276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.611472 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.612272 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.612277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.612491 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.612639 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.612939 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7083,"mutex_wait_us":45,"thread_start_us":6924,"threads_started":1}
I20260807 22:50:39.614104 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.581334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.9.194.254:39591)
0807 22:50:39.581588 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.581595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.596693 (+ 15098us) server_negotiation.cc:407] Connection header received
0807 22:50:39.596876 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.596881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.596964 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.597065 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.598778 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.599949 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.602460 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.603111 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.603232 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.605935 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.605944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.605949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.605990 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.609549 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.610865 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.610870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.610873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.611069 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.611575 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.611581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.611584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.612149 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.613282 (+ 1133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.613479 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.613873 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:50:39.615231 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16692
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.254@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.193@KRBTEST.COM
I20260807 22:50:39.634593 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.619355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.0.0.1:41828)
0807 22:50:39.619573 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.619591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.619698 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.620045 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.620051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.620541 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.620735 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.620747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.622176 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.622183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.623516 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.623528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.624289 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.624295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.624458 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.625025 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.625052 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.628091 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.632010 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.632015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.632019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.632513 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.632919 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.632924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.632927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.633088 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.633604 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.633608 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.633772 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.633910 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.634198 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 22:50:39.634575 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.619302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.9.194.252:34959)
0807 22:50:39.619464 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.619469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.619640 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:50:39.619818 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.619821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.619897 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.620012 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.620899 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.622029 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.623646 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.624129 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.624273 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.628308 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.628314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.628319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.628359 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.631841 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.632639 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.632644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.632648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.632771 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.633192 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.633197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.633200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.633472 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.633639 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.634018 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.634221 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:50:39.636708 16629 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.194.252:34959: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4"
member_type: NON_VOTER
last_known_addr {
host: "127.9.194.252"
port: 34959
}
attrs {
promote: true
}
}
dest_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1"
cas_config_opid_index: 6
I20260807 22:50:39.637699 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.616890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.9.194.193:43437)
0807 22:50:39.617586 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.617597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.617630 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:50:39.617704 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.617713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.618005 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.618178 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.620332 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.621530 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.623210 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.624154 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.624865 (+ 711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.630090 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.630104 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.630122 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.630165 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.634237 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.635127 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.635137 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.635140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.635293 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.635740 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.635746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.635749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.636126 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.636440 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.636799 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.637192 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":309,"threads_started":1}
I20260807 22:50:39.637516 16629 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } }
I20260807 22:50:39.639832 16827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:39.640015 17295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:50:39.640441 17384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:39.640699 17232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 22:50:39.643677 17233 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.644373 17295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.646248 17384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:50:39.647229 17231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:39.647804 17233 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.194.252:34959
I20260807 22:50:39.648402 17235 master.cc:537] Master@127.9.194.252:34959 shutting down...
I20260807 22:50:39.648519 17324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.648797 17324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:39.649991 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.650283 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
W20260807 22:50:39.650563 16612 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 22:50:39.651265 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.651626 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
W20260807 22:50:39.654682 16612 consensus_peers.cc:487] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 -> Peer 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252:34959): Couldn't send request to peer 0ca4789937cb494bb1782b208b1bc7e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:50:39.660486 17235 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:39.660912 17235 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:39.661120 17235 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: stopping tablet replica
I20260807 22:50:39.678670 17235 master.cc:552] Master@127.9.194.252:34959 shutdown complete.
I20260807 22:50:39.678772 17235 master_runner.cc:305] Clearing existing system tablet
I20260807 22:50:39.679190 17235 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:39.679314 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:39.682402 17235 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:50:39.690214 17443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:39.690580 17235 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:39.690824 17235 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.005s
I20260807 22:50:39.690929 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "0ca4789937cb494bb1782b208b1bc7e4"
format_stamp: "Formatted at 2026-08-07 22:50:39 on dist-test-slave-5k54"
server_key: "0b0850888fce30010a045e9c44ef8fa4"
server_key_iv: "76c5f294443855729a5978432ea8ff7b"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.696080 17235 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:39.703047 17235 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:50:39.703203 17235 log.cc:1196] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 22:50:39.703862 17235 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Deleting consensus metadata
I20260807 22:50:39.704137 17235 master_runner.cc:315] Copying system tablet from 127.9.194.254:39591
I20260807 22:50:39.709244 17235 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: tablet copy: Beginning tablet copy session from remote peer at address 127.9.194.254:39591
I20260807 22:50:39.725571 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.710167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.9.194.254:39591)
0807 22:50:39.710421 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:39.710429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:39.710807 (+ 378us) server_negotiation.cc:407] Connection header received
0807 22:50:39.711005 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:39.711011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:39.711094 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:39.711197 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:39.712299 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.713435 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.715347 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.716064 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.716184 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.718537 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:39.718544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:39.718549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:39.718589 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:39.722408 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.723364 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.723370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.723374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.723551 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:39.723996 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:39.724001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:39.724005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:39.724344 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:39.724478 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:39.724999 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 22:50:39.725228 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:50:39.725694 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:39.710070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35036)
0807 22:50:39.710681 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:39.710702 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:39.710868 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:39.711256 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:39.711262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:39.711862 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:39.712114 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.712126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.713585 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.713591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.715187 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:39.715205 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:39.716220 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:39.716226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:39.716415 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:39.717184 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:39.717211 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:39.718316 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:39.722670 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.722675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.722680 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.723228 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.723666 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:39.723671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:39.723675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:39.723869 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:39.724481 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:39.724486 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:39.724687 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 22:50:39.724876 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 22:50:39.725205 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":306,"threads_started":1}
I20260807 22:50:39.726542 16650 tablet_copy_service.cc:140] P afc993b5d4de4bb08faad4f4eb8a55b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0ca4789937cb494bb1782b208b1bc7e4 ({username='oryx', principal='oryx/127.9.194.252@KRBTEST.COM'} at 127.0.0.1:35036)
I20260807 22:50:39.726720 16650 tablet_copy_service.cc:161] P afc993b5d4de4bb08faad4f4eb8a55b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0ca4789937cb494bb1782b208b1bc7e4 at {username='oryx', principal='oryx/127.9.194.252@KRBTEST.COM'} at 127.0.0.1:35036: session id = 0ca4789937cb494bb1782b208b1bc7e4-00000000000000000000000000000000
I20260807 22:50:39.730360 16650 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:50:39.732411 17235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:50:39.739594 17235 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: tablet copy: Starting download of 0 data blocks...
I20260807 22:50:39.739910 17235 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: tablet copy: Starting download of 1 WAL segments...
I20260807 22:50:39.742372 17235 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:50:39.746850 16650 tablet_copy_service.cc:342] P afc993b5d4de4bb08faad4f4eb8a55b1: Request end of tablet copy session 0ca4789937cb494bb1782b208b1bc7e4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.194.252@KRBTEST.COM'} at 127.0.0.1:35036
I20260807 22:50:39.746986 16650 tablet_copy_service.cc:433] P afc993b5d4de4bb08faad4f4eb8a55b1: ending tablet copy session 0ca4789937cb494bb1782b208b1bc7e4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0ca4789937cb494bb1782b208b1bc7e4
I20260807 22:50:39.759948 17235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:50:39.760984 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16692
W20260807 22:50:39.763571 17450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.773072 17451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:39.779745 17453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:39.786000 17235 server_base.cc:1034] running on GCE node
I20260807 22:50:39.786515 17235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:39.787240 17235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:39.787879 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:43437
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:39591
--webserver_port=36953
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:50:39.788442 17235 hybrid_clock.cc:648] HybridClock initialized: now 1786143039788378 us; error 49 us; skew 500 ppm
I20260807 22:50:39.788702 17235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:39 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:39.791026 17235 init.cc:377] Logged in from keytab as oryx/127.9.194.252@KRBTEST.COM (short username oryx)
I20260807 22:50:39.800699 17235 webserver.cc:460] Webserver started at http://127.9.194.252:43013/ using document root <none> and password file <none>
I20260807 22:50:39.801019 17235 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:39.801126 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:39.803722 17235 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:50:39.805574 17461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:39.806082 17235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 22:50:39.806211 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "0ca4789937cb494bb1782b208b1bc7e4"
format_stamp: "Formatted at 2026-08-07 22:50:39 on dist-test-slave-5k54"
server_key: "0b0850888fce30010a045e9c44ef8fa4"
server_key_iv: "76c5f294443855729a5978432ea8ff7b"
server_key_version: "encryptionkey@0"
I20260807 22:50:39.806419 17235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:39.834060 17235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:39.838387 17235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:39.863561 17235 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.252:34959
I20260807 22:50:39.863617 17519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.252:34959 every 8 connection(s)
I20260807 22:50:39.864662 17235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
I20260807 22:50:39.869990 17520 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:50:39.873751 17520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Bootstrap starting.
I20260807 22:50:39.893383 17520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:50:39.894016 17520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: Bootstrap complete.
I20260807 22:50:39.895313 17520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } }
I20260807 22:50:39.895676 17520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0ca4789937cb494bb1782b208b1bc7e4, State: Initialized, Role: LEARNER
I20260807 22:50:39.895944 17520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } }
I20260807 22:50:39.897049 17520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:39.896838 17523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:39.897308 17523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:50:39.905097 17535 catalog_manager.cc:1259] Loaded cluster ID: 607daf0a5a7b461e9a550e7203095e48
I20260807 22:50:39.905191 17535 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: loading cluster ID for follower catalog manager: success
I20260807 22:50:39.908674 17535 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: acquiring CA information for follower catalog manager: success
I20260807 22:50:39.910923 17535 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:50:40.063524 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.049580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.0.0.1:41840)
0807 22:50:40.049873 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.049895 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.050040 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.056385 (+ 6345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.056392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.056431 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:40.056617 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.056628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.058823 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.058831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.060781 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.060797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.062743 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.062749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.062966 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.062999 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.063089 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.063216 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260807 22:50:40.064036 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.049996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.9.194.252:34959)
0807 22:50:40.055942 (+ 5946us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.055951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.055977 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:50:40.056061 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.056069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.056177 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.056333 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:40.056806 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.058654 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.060970 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.062544 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.062686 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.062966 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.063140 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.063325 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5775,"thread_start_us":5606,"threads_started":1}
I20260807 22:50:40.065395 17493 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260807 22:50:40.066007 17523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc993b5d4de4bb08faad4f4eb8a55b1. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:40.066352 17523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: LEARNER
W20260807 22:50:40.164709 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:40.165051 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:40.165227 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:40.172533 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:40.172683 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:40.172863 17457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:40.182884 17457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:43437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=36953
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:40.184278 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:40.185652 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:40.198964 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:40.198911 17545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:40.198998 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:40.200060 17457 server_base.cc:1034] running on GCE node
I20260807 22:50:40.200620 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:40.201567 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:40.202790 17457 hybrid_clock.cc:648] HybridClock initialized: now 1786143040202747 us; error 50 us; skew 500 ppm
I20260807 22:50:40.203102 17457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:40.206377 17457 init.cc:377] Logged in from keytab as oryx/127.9.194.193@KRBTEST.COM (short username oryx)
I20260807 22:50:40.207892 17457 webserver.cc:460] Webserver started at http://127.9.194.193:36953/ using document root <none> and password file <none>
I20260807 22:50:40.208563 17457 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:40.208720 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:40.215305 17457 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:50:40.218082 17552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.219413 17457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:40.219635 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "2da55ca5ba27443c9abc68f0d333e203"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "36fcfee442047a5276c9747ca351d10d"
server_key_iv: "b802fc8efd22fef295d4e430576c5971"
server_key_version: "encryptionkey@0"
I20260807 22:50:40.220616 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:40.250289 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:40.254746 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:40.255026 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:40.256000 17457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:40.257390 17457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:40.257508 17457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.257660 17457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:40.257756 17457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.292604 17457 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:43437
I20260807 22:50:40.292690 17671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:43437 every 8 connection(s)
I20260807 22:50:40.294059 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:40.294518 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 17457
I20260807 22:50:40.294895 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16840
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.194@KRBTEST.COM
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:40.334658 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.296497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.9.194.194:39959)
0807 22:50:40.297585 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.297594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.297627 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:50:40.297743 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.297754 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.298011 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.308962 (+ 10951us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.309805 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.313444 (+ 3639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.317777 (+ 4333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.318924 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.320207 (+ 1283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.326450 (+ 6243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.326468 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.326485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.326540 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.330772 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.331875 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.331885 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.331888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.332054 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.332534 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.332540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.332544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.333046 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.333353 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.333728 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.334017 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":718,"threads_started":1}
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
I20260807 22:50:40.336208 16974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:40.341001 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.298056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.9.194.193:52975)
0807 22:50:40.311563 (+ 13507us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.311738 (+ 175us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.315251 (+ 3513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.316582 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.316597 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.317509 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.318505 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.318533 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.320242 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.320250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.324026 (+ 3776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.324050 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.325560 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.325572 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.325893 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.326672 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.326704 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.329971 (+ 3267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.334241 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.334248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.334260 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.334685 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.335120 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.335124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.335127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.335463 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.338135 (+ 2672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.338144 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.338627 (+ 483us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.338931 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.339565 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12052,"thread_start_us":12175,"threads_started":1}
I20260807 22:50:40.341499 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.298421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:50441 (local address 127.9.194.254:39591)
0807 22:50:40.320756 (+ 22335us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.320762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.322683 (+ 1921us) server_negotiation.cc:407] Connection header received
0807 22:50:40.323673 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.323679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.323763 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.323884 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.325687 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.326839 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.329148 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.329806 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.329934 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.334308 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.334314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.334319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.334373 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.337670 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.338659 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.338668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.338673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.338882 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.339468 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.339476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.339482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.339932 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.340106 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.340767 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.341090 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22185,"thread_start_us":22352,"threads_started":1}
I20260807 22:50:40.342785 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.298381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:50441)
0807 22:50:40.321172 (+ 22791us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.321326 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.323524 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.324015 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.324021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.325057 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.325473 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.325537 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.326995 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.327000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.329003 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.329014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.330168 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.330175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.330390 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.331006 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.331028 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.334061 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.337804 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.337809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.337812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.338482 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.339062 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.339070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.339076 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.339266 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.340213 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.340220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.340428 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.341788 (+ 1360us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.342158 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21632,"spinlock_wait_cycles":19328,"thread_start_us":21458,"threads_started":1}
I20260807 22:50:40.343358 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.297779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:52975 (local address 127.9.194.252:34959)
0807 22:50:40.298712 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.298722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.311870 (+ 13148us) server_negotiation.cc:407] Connection header received
0807 22:50:40.315661 (+ 3791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.315670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.315775 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.315872 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.318725 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.320013 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.324302 (+ 4289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.325057 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.325181 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.330265 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.330273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.330280 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.330322 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.334079 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.334869 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.334874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.334877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.335022 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.335628 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.335635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.335640 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.337987 (+ 2347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.338159 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.342443 (+ 4284us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.342694 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755}
I20260807 22:50:40.344038 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.302207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.9.194.193:59437)
0807 22:50:40.323154 (+ 20947us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.323196 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.323339 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.323907 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.323911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.324975 (+ 1064us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.325358 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.325377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.326927 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.326933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.328726 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.328741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.329950 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.329957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.330150 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.331020 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.331038 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.337004 (+ 5966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.341055 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.341060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.341067 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.341574 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.342050 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.342054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.342057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.342228 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.342953 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.342958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.343095 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.343271 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.343569 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20806,"mutex_wait_us":3,"spinlock_wait_cycles":29440,"thread_start_us":20643,"threads_started":1}
I20260807 22:50:40.344333 17673 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:40.344551 17672 heartbeater.cc:347] Connected to a master server at 127.9.194.252:34959
I20260807 22:50:40.344674 17673 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.344741 17672 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.345927 17673 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:40.346378 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.302763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:59437 (local address 127.9.194.253:37461)
0807 22:50:40.320763 (+ 18000us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.320772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.323302 (+ 2530us) server_negotiation.cc:407] Connection header received
0807 22:50:40.323482 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.323488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.323591 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.323693 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.325569 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.326752 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.328903 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.329544 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.329662 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.337259 (+ 7597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.337267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.337273 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.337321 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.340891 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.341720 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.341727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.341730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.341905 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.342366 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.342371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.342373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.342713 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.342878 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.345710 (+ 2832us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.345955 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17848,"thread_start_us":17736,"threads_started":1}
I20260807 22:50:40.346935 17672 heartbeater.cc:511] Master 127.9.194.252:34959 requested a full tablet report, sending...
I20260807 22:50:40.347522 17674 heartbeater.cc:347] Connected to a master server at 127.9.194.253:37461
I20260807 22:50:40.347645 17674 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.348042 16629 ts_manager.cc:194] Re-registered known tserver with Master: 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:40.348073 17674 heartbeater.cc:511] Master 127.9.194.253:37461 requested a full tablet report, sending...
I20260807 22:50:40.348711 17481 ts_manager.cc:194] Registered new tserver with Master: 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:40.348948 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:50441
I20260807 22:50:40.349802 17285 ts_manager.cc:194] Registered new tserver with Master: 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:40.433167 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16840
I20260807 22:50:40.451575 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:39959
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:39591
--webserver_port=38217
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:50:40.503268 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: NON_VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: true } } }
I20260807 22:50:40.503722 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:40.505429 17686 raft_consensus.cc:1050] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: attempting to promote NON_VOTER 0ca4789937cb494bb1782b208b1bc7e4 to VOTER
I20260807 22:50:40.506054 17686 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } }
I20260807 22:50:40.508203 17493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 LEARNER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 22:50:40.508213 17295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Refusing update from remote peer afc993b5d4de4bb08faad4f4eb8a55b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 22:50:40.508774 17687 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 22:50:40.509227 17231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 22:50:40.511704 17231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.514385 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.514674 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:40.514703 17686 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.515118 17686 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:40.515547 17295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.519659 17690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.520025 17690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:40.521276 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.521584 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:40.521956 17493 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.525101 17523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:40.525357 17523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:50:40.785519 17685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:40.785779 17685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:40.785880 17685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:40.792976 17685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:40.793076 17685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:40.793183 17685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:40.803123 17685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:39959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=38217
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:40.804332 17685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:40.805652 17685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:40.819594 17695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:40.819567 17696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:40.820075 17698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:40.820729 17685 server_base.cc:1034] running on GCE node
I20260807 22:50:40.821290 17685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:40.822242 17685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:40.823531 17685 hybrid_clock.cc:648] HybridClock initialized: now 1786143040823484 us; error 37 us; skew 500 ppm
I20260807 22:50:40.823895 17685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:40.827296 17685 init.cc:377] Logged in from keytab as oryx/127.9.194.194@KRBTEST.COM (short username oryx)
I20260807 22:50:40.828745 17685 webserver.cc:460] Webserver started at http://127.9.194.194:38217/ using document root <none> and password file <none>
I20260807 22:50:40.829460 17685 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:40.829622 17685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:40.835492 17685 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:50:40.838270 17705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.839701 17685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:40.839881 17685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "1a18b33621054bb0b3e29e5f4b63b524"
format_stamp: "Formatted at 2026-08-07 22:50:37 on dist-test-slave-5k54"
server_key: "7244380d6434b8afdca1acd1897ac463"
server_key_iv: "31abc4fb79b42f08f60b0067f3b0a8b7"
server_key_version: "encryptionkey@0"
I20260807 22:50:40.840879 17685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:40.860298 17685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:40.865095 17685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:40.865397 17685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:40.866482 17685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:40.868177 17685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:40.868289 17685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.868392 17685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:40.868468 17685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:40.910674 17685 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:39959
I20260807 22:50:40.910769 17825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:39959 every 8 connection(s)
I20260807 22:50:40.912204 17685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:40.914706 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 17685
I20260807 22:50:40.915041 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16986
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
Aug 07 22:50:40 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.195@KRBTEST.COM
I20260807 22:50:40.965539 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.934615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:40081 (local address 127.9.194.254:39591)
0807 22:50:40.941300 (+ 6685us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.941305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.945353 (+ 4048us) server_negotiation.cc:407] Connection header received
0807 22:50:40.945428 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.945433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.945544 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.945656 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.947725 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.948833 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.951603 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.952243 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952358 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.957981 (+ 5623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.957997 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.958003 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.958047 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.961934 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.962712 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.962720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.962724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.962887 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.963316 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.963320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.963324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.963647 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.963822 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.964945 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.965172 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6584,"thread_start_us":6499,"threads_started":1}
I20260807 22:50:40.966790 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.934679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:40081)
0807 22:50:40.944733 (+ 10054us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.944755 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.944931 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.945705 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.945712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.946691 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.947558 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.947583 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.948981 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.948987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.951342 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.951359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.952424 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.952826 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.953964 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.953988 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.957756 (+ 3768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.962075 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.962084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.962101 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.962591 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.962996 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.963003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.963007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.963197 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.964247 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.964252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.964543 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.964836 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.965347 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9903,"mutex_wait_us":376,"spinlock_wait_cycles":35840,"thread_start_us":9726,"threads_started":1}
I20260807 22:50:40.968287 17827 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:40.968357 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.916173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.9.194.194:52151)
0807 22:50:40.943909 (+ 27736us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.943930 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.944081 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.945184 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.945196 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.946475 (+ 1279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.947582 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.947599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.949112 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.949121 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.951355 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.951370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.952644 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.952836 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.953800 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.953830 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.958558 (+ 4728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.965367 (+ 6809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.965372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.965375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.965937 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.966361 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.966364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.966367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.966562 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.967277 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.967282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.967468 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.967636 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.967916 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27617,"mutex_wait_us":626,"spinlock_wait_cycles":6016,"thread_start_us":24893,"threads_started":1}
I20260807 22:50:40.968654 17827 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.969240 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.916325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.9.194.195:43887)
0807 22:50:40.945648 (+ 29323us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.945658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.945685 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:50:40.945759 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.945768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.946033 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.946214 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.949899 (+ 3685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.951410 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.954208 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.955114 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.955235 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.960797 (+ 5562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.960816 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.960840 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.960883 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.964984 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.965835 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.965844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.965849 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.966043 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.966510 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.966515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.966517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.966944 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.967236 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.968372 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.968792 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29155,"thread_start_us":28979,"threads_started":1}
I20260807 22:50:40.969455 17827 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:40.970561 17119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:40.970657 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.916170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:36297 (local address 127.9.194.252:34959)
0807 22:50:40.939568 (+ 23398us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.939579 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.941962 (+ 2383us) server_negotiation.cc:407] Connection header received
0807 22:50:40.944324 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.944332 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.944422 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.944522 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.947767 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.948585 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.951545 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.951990 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952063 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.959639 (+ 7576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.959647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.959653 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.959692 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.963631 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.965041 (+ 1410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.965049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.965055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.965220 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.968923 (+ 3703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.968929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.968932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.969295 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.969455 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.969993 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.970227 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23175,"thread_start_us":23418,"threads_started":1}
I20260807 22:50:40.970885 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.915084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:52151 (local address 127.9.194.253:37461)
0807 22:50:40.940134 (+ 25050us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:40.940144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:40.944035 (+ 3891us) server_negotiation.cc:407] Connection header received
0807 22:50:40.944618 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:40.944624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:40.944721 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:40.944916 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:40.947765 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.948898 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.951556 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.952224 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952332 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.958791 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:40.958799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:40.958807 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:40.958853 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:40.964862 (+ 6009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.966079 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.966088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.966093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.966246 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:40.966702 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:40.966709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:40.966712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:40.967060 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:40.967192 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:40.970198 (+ 3006us) server_negotiation.cc:299] Negotiation successful
0807 22:50:40.970431 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24882,"thread_start_us":25044,"threads_started":1}
I20260807 22:50:40.970955 16629 ts_manager.cc:194] Re-registered known tserver with Master: 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959)
I20260807 22:50:40.971592 17828 heartbeater.cc:347] Connected to a master server at 127.9.194.253:37461
I20260807 22:50:40.971728 17828 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.971900 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:40.915751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.9.194.194:36297)
0807 22:50:40.941770 (+ 26019us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:40.941820 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:40.943891 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:40.945184 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:40.945196 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:40.946502 (+ 1306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:40.947564 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.947583 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.948766 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.948776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.951369 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:40.951378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:40.952407 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:40.952414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:40.952823 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:40.953829 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:40.953863 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:40.959362 (+ 5499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:40.963773 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.963779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.963782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.964149 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.968584 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:40.968589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:40.968592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:40.968789 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:40.969544 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:40.969548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:40.969706 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 22:50:40.970868 (+ 1162us) client_negotiation.cc:240] Negotiation successful
0807 22:50:40.971268 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25753,"mutex_wait_us":539,"spinlock_wait_cycles":28032,"thread_start_us":27977,"threads_started":1}
I20260807 22:50:40.972028 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:40081
I20260807 22:50:40.972281 17828 heartbeater.cc:511] Master 127.9.194.253:37461 requested a full tablet report, sending...
I20260807 22:50:40.972892 17826 heartbeater.cc:347] Connected to a master server at 127.9.194.252:34959
I20260807 22:50:40.973043 17826 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:40.973522 17285 ts_manager.cc:194] Registered new tserver with Master: 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959)
I20260807 22:50:40.973541 17826 heartbeater.cc:511] Master 127.9.194.252:34959 requested a full tablet report, sending...
I20260807 22:50:40.974879 17481 ts_manager.cc:194] Registered new tserver with Master: 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959)
I20260807 22:50:41.060465 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16986
I20260807 22:50:41.079281 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:43887
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:39591
--webserver_port=33863
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:50:41.352811 17673 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
W20260807 22:50:41.415344 17840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:41.415774 17840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:41.415889 17840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:41.423342 17840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:41.423523 17840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:41.423720 17840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:41.433694 17840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:43887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=33863
--tserver_master_addrs=127.9.194.254:39591,127.9.194.253:37461,127.9.194.252:34959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:41.434937 17840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:41.436239 17840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:41.449488 17848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:41.449584 17845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:41.449565 17846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:41.450310 17840 server_base.cc:1034] running on GCE node
I20260807 22:50:41.450909 17840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:41.451910 17840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:41.453109 17840 hybrid_clock.cc:648] HybridClock initialized: now 1786143041453052 us; error 50 us; skew 500 ppm
I20260807 22:50:41.453375 17840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:41 dist-test-slave-5k54 krb5kdc[16575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143041, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:41.456709 17840 init.cc:377] Logged in from keytab as oryx/127.9.194.195@KRBTEST.COM (short username oryx)
I20260807 22:50:41.458175 17840 webserver.cc:460] Webserver started at http://127.9.194.195:33863/ using document root <none> and password file <none>
I20260807 22:50:41.458895 17840 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:41.459041 17840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:41.464327 17840 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:50:41.467232 17855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:41.468677 17840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:50:41.468853 17840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "517a04892fdf40c0a9a582ba9c273cd8"
format_stamp: "Formatted at 2026-08-07 22:50:38 on dist-test-slave-5k54"
server_key: "a13accac299dbad704c22a1f12345785"
server_key_iv: "b1c15737362f056b438807115f954b8b"
server_key_version: "encryptionkey@0"
I20260807 22:50:41.469839 17840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:41.493280 17840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:41.497485 17840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:41.497762 17840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:41.498986 17840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:41.500591 17840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:41.500716 17840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:41.500880 17840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:41.500979 17840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:41.535096 17840 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:43887
I20260807 22:50:41.535195 17977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:43887 every 8 connection(s)
I20260807 22:50:41.536695 17840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:41.543624 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 17840
Aug 07 22:50:41 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143041, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.253@KRBTEST.COM
Aug 07 22:50:41 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143041, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
Aug 07 22:50:41 dist-test-slave-5k54 krb5kdc[16575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143041, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.252@KRBTEST.COM
I20260807 22:50:41.600596 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.541222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:51741 (local address 127.9.194.253:37461)
0807 22:50:41.541668 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.541673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.549177 (+ 7504us) server_negotiation.cc:407] Connection header received
0807 22:50:41.552564 (+ 3387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.552574 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.552691 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.552793 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.555893 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.557430 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.562893 (+ 5463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.563878 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.564005 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.582536 (+ 18531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.582547 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.582555 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.582611 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.587709 (+ 5098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.590406 (+ 2697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.590416 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.590422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.590630 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.594083 (+ 3453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.594091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.594097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.594568 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.594762 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.599530 (+ 4768us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.599841 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":249,"threads_started":1}
I20260807 22:50:41.603788 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.541524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.9.194.195:51741)
0807 22:50:41.549045 (+ 7521us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.549069 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.552114 (+ 3045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.553525 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.553541 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.554565 (+ 1024us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:41.555630 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.555661 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.557704 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.557712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.560474 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.560492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.565721 (+ 5229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.565730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.566133 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.573270 (+ 7137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:41.573349 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:41.578667 (+ 5318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:41.587978 (+ 9311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.587994 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.588021 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.588863 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.590903 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.590911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.590917 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.591284 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.595929 (+ 4645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:41.595941 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:41.596567 (+ 626us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.601140 (+ 4573us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.601924 (+ 784us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7365,"spinlock_wait_cycles":5888,"thread_start_us":7948,"threads_started":1}
I20260807 22:50:41.607483 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.540083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:41775 (local address 127.9.194.252:34959)
0807 22:50:41.540511 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.540515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.547555 (+ 7040us) server_negotiation.cc:407] Connection header received
0807 22:50:41.560822 (+ 13267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.560831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.560946 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.561049 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.566693 (+ 5644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.568350 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.575287 (+ 6937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.576409 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.576536 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.596989 (+ 20453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.596999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.597007 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.597065 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.602020 (+ 4955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.604943 (+ 2923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.604948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.604951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.605137 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.605598 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.605603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.605606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.606016 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.606144 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.606756 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.607031 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":723,"threads_started":1}
I20260807 22:50:41.608438 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.539774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:53559 (local address 127.9.194.254:39591)
0807 22:50:41.540235 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.540244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.549063 (+ 8819us) server_negotiation.cc:407] Connection header received
0807 22:50:41.562857 (+ 13794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.562866 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.562979 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.563113 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.565509 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.567292 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.573061 (+ 5769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.573897 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.574048 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.592644 (+ 18596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.592651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.592655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.592704 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.597746 (+ 5042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.604984 (+ 7238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.604994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.605001 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.605185 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.606703 (+ 1518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.606709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.606714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.607100 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.607261 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.607842 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.608081 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":424,"threads_started":1}
I20260807 22:50:41.608566 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.540318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.9.194.195:41775)
0807 22:50:41.547298 (+ 6980us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.547373 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.560628 (+ 13255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.561341 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.561348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.562142 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:41.562538 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.562554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.568576 (+ 6022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.568585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.570649 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.570667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.576890 (+ 6223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.576899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.577166 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.579697 (+ 2531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:41.579729 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:41.593359 (+ 13630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:41.603573 (+ 10214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.603581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.603588 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.604755 (+ 1167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.605291 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.605296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.605300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.605457 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.606246 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:41.606254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:41.606432 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.607701 (+ 1269us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.608115 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6572,"thread_start_us":692,"threads_started":1}
I20260807 22:50:41.610430 17980 heartbeater.cc:347] Connected to a master server at 127.9.194.253:37461
I20260807 22:50:41.610628 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.540538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:53559)
0807 22:50:41.548944 (+ 8406us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.548963 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.562700 (+ 13737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.564272 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.564279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.564990 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:41.565350 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.565366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.570836 (+ 5470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.570843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.572895 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.572912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.578981 (+ 6069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.578990 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.579243 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.581392 (+ 2149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:41.581423 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:41.592402 (+ 10979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:41.598207 (+ 5805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.598214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.598221 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.598908 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.605742 (+ 6834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:41.605746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:41.605750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:41.605906 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:41.607225 (+ 1319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:41.607229 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:41.607389 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.607559 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.608398 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8282,"spinlock_wait_cycles":106496,"thread_start_us":8063,"threads_started":1}
I20260807 22:50:41.610795 17980 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:41.611567 17979 heartbeater.cc:347] Connected to a master server at 127.9.194.254:39591
I20260807 22:50:41.611706 17979 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:41.611702 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.550850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.9.194.253:37461)
0807 22:50:41.579927 (+ 29077us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.579936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.580926 (+ 990us) server_negotiation.cc:407] Connection header received
0807 22:50:41.581008 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.581016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.581120 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.581273 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.584385 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.585891 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.587827 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.588487 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.588589 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.592877 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.592887 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.592894 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.592931 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.605792 (+ 12861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.607363 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.607369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.607372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.607580 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.608656 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.608661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.608664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.609032 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.610971 (+ 1939us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.611094 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.611267 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28910,"thread_start_us":29094,"threads_started":1}
I20260807 22:50:41.611902 17980 heartbeater.cc:511] Master 127.9.194.253:37461 requested a full tablet report, sending...
I20260807 22:50:41.612244 17979 heartbeater.cc:511] Master 127.9.194.254:39591 requested a full tablet report, sending...
I20260807 22:50:41.612497 17978 heartbeater.cc:347] Connected to a master server at 127.9.194.252:34959
I20260807 22:50:41.612643 17978 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:41.613113 17978 heartbeater.cc:511] Master 127.9.194.252:34959 requested a full tablet report, sending...
I20260807 22:50:41.613827 17285 ts_manager.cc:194] Registered new tserver with Master: 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:41.613968 16629 ts_manager.cc:194] Re-registered known tserver with Master: 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:41.614357 17481 ts_manager.cc:194] Registered new tserver with Master: 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:41.615111 16629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:53559
I20260807 22:50:41.628165 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.549639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.9.194.252:34959)
0807 22:50:41.604698 (+ 55059us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.604702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.604718 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:41.604762 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.604765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.604839 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.608397 (+ 3558us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.609052 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.610229 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.617190 (+ 6961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.617802 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.617948 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.621824 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.621830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.621833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.621862 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.625289 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.625987 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.625991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.625994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.626126 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.626536 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.626541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.626543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.626945 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.627156 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.627503 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.627748 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54942,"thread_start_us":54802,"threads_started":1}
I20260807 22:50:41.628710 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.574445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.9.194.254:39591)
0807 22:50:41.594373 (+ 19928us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.594381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.605535 (+ 11154us) server_negotiation.cc:407] Connection header received
0807 22:50:41.605724 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.605729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.605805 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.605901 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:41.615744 (+ 9843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.616943 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.618730 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.619149 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.619244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.622315 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:41.622323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:41.622329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:41.622372 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:41.625768 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.626531 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.626535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.626539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.626716 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:41.627123 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:41.627128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:41.627131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:41.627509 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:41.627789 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.628120 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.628353 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19777,"thread_start_us":30948,"threads_started":1}
I20260807 22:50:41.632881 16629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:41.639973 16629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:41.681727 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.659042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.9.194.193:43437)
0807 22:50:41.674260 (+ 15218us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.674268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.674291 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:41.674372 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.674378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.674617 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.674762 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:41.675562 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.677186 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.679626 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.680569 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.680716 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.680972 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.681260 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.681414 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15059,"thread_start_us":14934,"threads_started":1}
I20260807 22:50:41.681779 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.658708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43437 (local address 127.0.0.1:38940)
0807 22:50:41.673517 (+ 14809us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.673544 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.673714 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.674773 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.674779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.674808 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:41.675286 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.675301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.677306 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.677310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.679466 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.679480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.680777 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.680784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.681070 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.681096 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.681195 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.681322 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14615,"mutex_wait_us":58,"thread_start_us":16194,"threads_started":1}
I20260807 22:50:41.683393 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.659312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.9.194.194:39959)
0807 22:50:41.672686 (+ 13374us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.672700 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.673741 (+ 1041us) server_negotiation.cc:407] Connection header received
0807 22:50:41.673937 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.673944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.674211 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.674480 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:41.677004 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.678082 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.680514 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.682034 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.682132 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.682297 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.682952 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.683110 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13104,"thread_start_us":13379,"threads_started":1}
I20260807 22:50:41.684363 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.659075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39959 (local address 127.0.0.1:51610)
0807 22:50:41.673599 (+ 14524us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.673620 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.673769 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.674416 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.674420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.674448 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:41.674842 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.674857 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.678239 (+ 3382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.678242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.680335 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.680350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.682552 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.682560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.682736 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.682769 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.683676 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.683810 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14383,"mutex_wait_us":21,"thread_start_us":14204,"threads_started":1}
I20260807 22:50:41.685072 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.675187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43887 (local address 127.0.0.1:51598)
0807 22:50:41.675812 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.675831 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.675942 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.676614 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.676619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.676642 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:41.677069 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.677083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.679109 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.679115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.681375 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.681389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.684192 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.684198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.684396 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.684426 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.684517 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.684646 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":38,"thread_start_us":356,"threads_started":1}
I20260807 22:50:41.685557 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.675496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.9.194.195:43887)
0807 22:50:41.676185 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.676192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.676216 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:50:41.676289 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.676293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.676434 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.676627 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:41.677266 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.678958 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.681622 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.683981 (+ 2359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.684155 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.684722 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.684832 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.684982 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":340,"threads_started":1}
I20260807 22:50:41.685962 17601 tablet_service.cc:1467] Processing CreateTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 (DEFAULT_TABLE table=test-table [id=5db4abfa0f3248e4b082c5acc83458db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:41.687544 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16a907cf2d24ce7a90dfd300df0a818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:41.688012 17758 tablet_service.cc:1467] Processing CreateTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 (DEFAULT_TABLE table=test-table [id=5db4abfa0f3248e4b082c5acc83458db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:41.688401 17910 tablet_service.cc:1467] Processing CreateTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 (DEFAULT_TABLE table=test-table [id=5db4abfa0f3248e4b082c5acc83458db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:41.689599 17910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16a907cf2d24ce7a90dfd300df0a818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:41.690464 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16a907cf2d24ce7a90dfd300df0a818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:41.702354 18021 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Bootstrap starting.
I20260807 22:50:41.703907 18023 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Bootstrap starting.
I20260807 22:50:41.705969 18021 tablet_bootstrap.cc:654] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:41.707096 18021 log.cc:824] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:41.707677 18023 tablet_bootstrap.cc:654] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:41.708267 18024 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Bootstrap starting.
I20260807 22:50:41.708951 18023 log.cc:824] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:41.712054 18023 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: No bootstrap required, opened a new log
I20260807 22:50:41.712253 18024 tablet_bootstrap.cc:654] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:41.712339 18023 ts_tablet_manager.cc:1397] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 22:50:41.712965 18021 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: No bootstrap required, opened a new log
I20260807 22:50:41.713245 18021 ts_tablet_manager.cc:1397] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 22:50:41.713491 18024 log.cc:824] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:41.716475 18024 tablet_bootstrap.cc:492] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: No bootstrap required, opened a new log
I20260807 22:50:41.716725 18024 ts_tablet_manager.cc:1397] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:50:41.720598 18023 raft_consensus.cc:348] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.720970 18023 raft_consensus.cc:374] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:41.721077 18023 raft_consensus.cc:729] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da55ca5ba27443c9abc68f0d333e203, State: Initialized, Role: FOLLOWER
I20260807 22:50:41.721786 18023 consensus_queue.cc:260] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.725136 18023 ts_tablet_manager.cc:1428] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 22:50:41.723819 18021 raft_consensus.cc:348] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.725826 18021 raft_consensus.cc:374] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:41.725925 18021 raft_consensus.cc:729] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517a04892fdf40c0a9a582ba9c273cd8, State: Initialized, Role: FOLLOWER
I20260807 22:50:41.726598 18021 consensus_queue.cc:260] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.727337 18024 raft_consensus.cc:348] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.727895 18024 raft_consensus.cc:374] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:41.728018 18024 raft_consensus.cc:729] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a18b33621054bb0b3e29e5f4b63b524, State: Initialized, Role: FOLLOWER
I20260807 22:50:41.728789 18024 consensus_queue.cc:260] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.734562 17979 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
I20260807 22:50:41.735713 18021 ts_tablet_manager.cc:1428] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Time spent starting tablet: real 0.022s user 0.010s sys 0.011s
I20260807 22:50:41.736922 18024 ts_tablet_manager.cc:1428] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260807 22:50:41.739235 17827 heartbeater.cc:503] Master 127.9.194.254:39591 was elected leader, sending a full tablet report...
W20260807 22:50:41.790287 17981 tablet.cc:2369] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:50:41.823204 17683 tablet.cc:2369] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:41.862389 18031 raft_consensus.cc:482] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:41.862669 18031 raft_consensus.cc:504] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.864759 18031 leader_election.cc:288] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437), 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:41.872610 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.865276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:58677 (local address 127.9.194.193:43437)
0807 22:50:41.865578 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.865585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.865900 (+ 315us) server_negotiation.cc:407] Connection header received
0807 22:50:41.866189 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.866195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.866268 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.866325 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:41.867164 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.868253 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.870654 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.871598 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.871809 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.871893 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.872191 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.872332 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 22:50:41.872653 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.865119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43437 (local address 127.9.194.194:58677)
0807 22:50:41.865789 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.865807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.866021 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.866526 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.866531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.866556 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:41.866972 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.866985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.868406 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.868412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.870468 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.870481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.871810 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.871816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.872011 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.872038 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.872151 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.872297 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":341,"threads_started":1}
I20260807 22:50:41.874138 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.866100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:41351 (local address 127.9.194.195:43887)
0807 22:50:41.866400 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.866409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.867116 (+ 707us) server_negotiation.cc:407] Connection header received
0807 22:50:41.867343 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.867348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.867413 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.867568 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:41.868148 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.869361 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.871772 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.873014 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.873166 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.873234 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.873712 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.873856 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:50:41.874261 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.865951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43887 (local address 127.9.194.194:41351)
0807 22:50:41.866983 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:41.866999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:41.867167 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:41.867599 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:41.867604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:41.867633 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:41.867964 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.867973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.869508 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.869513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.871602 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:41.871616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.873388 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.873396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:41.873540 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.873565 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:50:41.873682 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 22:50:41.873822 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":705,"threads_started":1}
I20260807 22:50:41.874183 17623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16a907cf2d24ce7a90dfd300df0a818" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da55ca5ba27443c9abc68f0d333e203" is_pre_election: true
I20260807 22:50:41.874788 17623 raft_consensus.cc:2454] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 0.
I20260807 22:50:41.875285 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16a907cf2d24ce7a90dfd300df0a818" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517a04892fdf40c0a9a582ba9c273cd8" is_pre_election: true
I20260807 22:50:41.875622 17709 leader_election.cc:302] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18b33621054bb0b3e29e5f4b63b524, 2da55ca5ba27443c9abc68f0d333e203; no voters:
I20260807 22:50:41.875896 17930 raft_consensus.cc:2454] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 0.
I20260807 22:50:41.876068 18031 raft_consensus.cc:2781] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:41.876240 18031 raft_consensus.cc:482] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:41.876361 18031 raft_consensus.cc:3037] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:41.879938 18031 raft_consensus.cc:504] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.880630 18031 leader_election.cc:288] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 election: Requested vote from peers 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437), 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887)
I20260807 22:50:41.881070 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16a907cf2d24ce7a90dfd300df0a818" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517a04892fdf40c0a9a582ba9c273cd8"
I20260807 22:50:41.881053 17623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16a907cf2d24ce7a90dfd300df0a818" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da55ca5ba27443c9abc68f0d333e203"
I20260807 22:50:41.881320 17930 raft_consensus.cc:3037] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:41.881280 17623 raft_consensus.cc:3037] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:41.883832 17930 raft_consensus.cc:2454] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 1.
I20260807 22:50:41.884238 17709 leader_election.cc:302] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18b33621054bb0b3e29e5f4b63b524, 517a04892fdf40c0a9a582ba9c273cd8; no voters:
I20260807 22:50:41.884527 18031 raft_consensus.cc:2781] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:41.884572 17623 raft_consensus.cc:2454] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 1.
I20260807 22:50:41.885110 18031 raft_consensus.cc:686] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 LEADER]: Becoming Leader. State: Replica: 1a18b33621054bb0b3e29e5f4b63b524, State: Running, Role: LEADER
I20260807 22:50:41.885829 18031 consensus_queue.cc:237] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } }
I20260807 22:50:41.890937 16629 catalog_manager.cc:5495] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "1a18b33621054bb0b3e29e5f4b63b524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } health_report { overall_health: UNKNOWN } } }
W20260807 22:50:41.941964 17829 tablet.cc:2369] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:41.964373 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:41.959535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.9.194.194:39959)
0807 22:50:41.959716 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:41.959721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:41.959737 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:41.959827 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:41.959830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:41.959920 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:41.960048 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:41.960380 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.961093 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.962439 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:41.962828 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:41.962926 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:41.963335 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:41.963883 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:41.963997 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:41.964104 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 22:50:41.964169 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:50:41.972515 17930 raft_consensus.cc:1261] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Refusing update from remote peer 1a18b33621054bb0b3e29e5f4b63b524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:41.972637 17623 raft_consensus.cc:1261] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Refusing update from remote peer 1a18b33621054bb0b3e29e5f4b63b524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:41.973513 18034 consensus_queue.cc:1035] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:41.973889 18031 consensus_queue.cc:1035] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:41.987730 18037 mvcc.cc:205] Tried to move back new op lower bound from 7316041899909849088 to 7316041899567890432. Current Snapshot: MvccSnapshot[applied={T|T < 7316041899909849088}]
I20260807 22:50:41.989344 18040 mvcc.cc:205] Tried to move back new op lower bound from 7316041899909849088 to 7316041899567890432. Current Snapshot: MvccSnapshot[applied={T|T < 7316041899909849088}]
I20260807 22:50:42.007963 16628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:42.008324 16628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:42.014468 16628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afc993b5d4de4bb08faad4f4eb8a55b1: Sending DeleteTablet for 3 replicas of tablet a16a907cf2d24ce7a90dfd300df0a818
I20260807 22:50:42.016212 17758 tablet_service.cc:1514] Processing DeleteTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:42 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51610
I20260807 22:50:42.016999 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:42 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:38940
I20260807 22:50:42.017887 17910 tablet_service.cc:1514] Processing DeleteTablet for tablet a16a907cf2d24ce7a90dfd300df0a818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:42 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51598
I20260807 22:50:42.018693 18051 tablet_replica.cc:330] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: stopping tablet replica
I20260807 22:50:42.019357 18051 raft_consensus.cc:2229] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:42.019959 18051 raft_consensus.cc:2258] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:42.022447 18051 ts_tablet_manager.cc:1905] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:42.026081 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 16591
I20260807 22:50:42.029064 16661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:42.032311 18051 ts_tablet_manager.cc:1918] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:42.032560 18051 log.cc:1196] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/a16a907cf2d24ce7a90dfd300df0a818
I20260807 22:50:42.033167 18051 ts_tablet_manager.cc:1939] T a16a907cf2d24ce7a90dfd300df0a818 P 517a04892fdf40c0a9a582ba9c273cd8: Deleting consensus metadata
I20260807 22:50:42.036458 18049 tablet_replica.cc:330] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: stopping tablet replica
I20260807 22:50:42.036960 18049 raft_consensus.cc:2229] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:42.037551 18049 raft_consensus.cc:2258] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:42.038545 18048 tablet_replica.cc:330] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: stopping tablet replica
I20260807 22:50:42.039065 18048 raft_consensus.cc:2229] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:42.039945 18048 raft_consensus.cc:2258] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:42.042975 18048 ts_tablet_manager.cc:1905] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:42.045787 18049 ts_tablet_manager.cc:1905] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:42.050737 18048 ts_tablet_manager.cc:1918] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:42.050931 18048 log.cc:1196] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/a16a907cf2d24ce7a90dfd300df0a818
I20260807 22:50:42.051590 18048 ts_tablet_manager.cc:1939] T a16a907cf2d24ce7a90dfd300df0a818 P 1a18b33621054bb0b3e29e5f4b63b524: Deleting consensus metadata
I20260807 22:50:42.052922 18049 ts_tablet_manager.cc:1918] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:42.053088 18049 log.cc:1196] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/a16a907cf2d24ce7a90dfd300df0a818
I20260807 22:50:42.053673 18049 ts_tablet_manager.cc:1939] T a16a907cf2d24ce7a90dfd300df0a818 P 2da55ca5ba27443c9abc68f0d333e203: Deleting consensus metadata
I20260807 22:50:42.056265 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:42.017627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.9.194.253:37461)
0807 22:50:42.017880 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:42.017888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:42.024918 (+ 7030us) server_negotiation.cc:407] Connection header received
0807 22:50:42.025826 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:42.025833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:42.025949 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:42.026026 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:42.028942 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.030063 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.032066 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.032746 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.032868 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:42.036741 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:42.036748 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:42.036753 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:42.036794 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:42.040751 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.043033 (+ 2282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.043040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.043044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.043197 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.044122 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.044127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.044130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.044506 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:42.044749 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:42.055694 (+ 10945us) server_negotiation.cc:299] Negotiation successful
0807 22:50:42.055964 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:50:42.073288 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:42.024970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.9.194.252:34959)
0807 22:50:42.025156 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:42.025162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:42.056471 (+ 31309us) server_negotiation.cc:407] Connection header received
0807 22:50:42.056738 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:42.056746 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:42.056840 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:42.056912 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:42.058903 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.060180 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.062050 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.062808 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.062958 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:42.066492 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:42.066498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:42.066503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:42.066543 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:42.070300 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.071155 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.071160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.071164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.071302 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.071768 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.071773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.071776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.072125 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:42.072374 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:42.072730 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 22:50:42.072955 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:50:42.172154 16610 catalog_manager.cc:4841] TS 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959): tablet a16a907cf2d24ce7a90dfd300df0a818 (table test-table [id=5db4abfa0f3248e4b082c5acc83458db]) successfully deleted
I20260807 22:50:42.172942 16612 catalog_manager.cc:4841] TS 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887): tablet a16a907cf2d24ce7a90dfd300df0a818 (table test-table [id=5db4abfa0f3248e4b082c5acc83458db]) successfully deleted
I20260807 22:50:42.173225 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 16591
W20260807 22:50:42.197199 17556 connection.cc:535] server connection from 127.0.0.1:38940 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:50:42.230948 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:42.202946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.9.194.252:34959)
0807 22:50:42.203126 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:42.203132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:42.213323 (+ 10191us) server_negotiation.cc:407] Connection header received
0807 22:50:42.213538 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:42.213544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:42.213644 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:42.213755 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:42.215371 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.216511 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.219003 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.219808 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.219905 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:42.224117 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:42.224124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:42.224128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:42.224175 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:42.227814 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.228651 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.228657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.228661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.228798 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.229235 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.229240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.229244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.229625 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:42.229849 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:42.230324 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 22:50:42.230565 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:50:42.231334 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:42.202454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.9.194.253:37461)
0807 22:50:42.202679 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:42.202687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:42.212476 (+ 9789us) server_negotiation.cc:407] Connection header received
0807 22:50:42.212683 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:42.212688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:42.212802 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:42.212936 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:42.214917 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.215984 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.218177 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:42.218838 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:42.218989 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:42.223355 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:42.223369 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:42.223373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:42.223416 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:42.227209 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.228077 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.228085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.228091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.228261 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:42.228970 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:42.228978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:42.228982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:42.229361 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:42.229612 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:42.230795 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0807 22:50:42.230970 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:50:42.998117 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:42.997322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:34687)
0807 22:50:42.997774 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:42.997922 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":225,"threads_started":1}
W20260807 22:50:42.998564 17827 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:39591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.009654 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.008553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:54407)
0807 22:50:43.009300 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.009449 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":532,"threads_started":1}
W20260807 22:50:43.010165 17673 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:39591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.011092 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.007885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:47097)
0807 22:50:43.010813 (+ 2928us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.010928 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2776,"thread_start_us":2674,"threads_started":1}
W20260807 22:50:43.011503 17979 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:39591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.515283 18063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader afc993b5d4de4bb08faad4f4eb8a55b1)
I20260807 22:50:43.515580 18063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } }
I20260807 22:50:43.516515 18063 leader_election.cc:288] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591), 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252:34959)
I20260807 22:50:43.517963 18067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader afc993b5d4de4bb08faad4f4eb8a55b1)
I20260807 22:50:43.518409 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.516642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35468)
0807 22:50:43.517250 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.517998 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
I20260807 22:50:43.518338 18067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } }
W20260807 22:50:43.518723 17269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.520399 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.519973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35478)
0807 22:50:43.520185 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.520264 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 22:50:43.520593 17269 leader_election.cc:334] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591): Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.520864 18067 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591), 72809db13eaa4248addf9720111c98c8 (127.9.194.253:37461)
I20260807 22:50:43.521869 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.520196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35488)
0807 22:50:43.521350 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.521588 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":910,"threads_started":1}
W20260807 22:50:43.522099 17466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.527590 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.516690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:34959 (local address 127.0.0.1:41870)
0807 22:50:43.518431 (+ 1741us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.518449 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:43.518601 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:43.519247 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:43.519255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:43.519292 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:43.519668 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.519677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.521390 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.521395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.523945 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.523968 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.526579 (+ 2611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.526588 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.526842 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.526893 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 22:50:43.527014 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:50:43.527201 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":1352,"threads_started":1}
I20260807 22:50:43.528587 18066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.516986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.9.194.252:34959)
0807 22:50:43.517580 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.517589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.518664 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 22:50:43.518755 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.518760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.518867 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.518975 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:43.519919 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.521199 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.524225 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.526080 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.526247 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.526416 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.527899 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.528118 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":272,"threads_started":1}
I20260807 22:50:43.530754 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72809db13eaa4248addf9720111c98c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ca4789937cb494bb1782b208b1bc7e4" is_pre_election: true
I20260807 22:50:43.531214 17493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72809db13eaa4248addf9720111c98c8 in term 1.
I20260807 22:50:43.531594 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.530812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35500)
0807 22:50:43.531015 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.531350 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:50:43.531826 17466 leader_election.cc:334] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591): Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.531773 17268 leader_election.cc:302] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ca4789937cb494bb1782b208b1bc7e4, 72809db13eaa4248addf9720111c98c8; no voters: afc993b5d4de4bb08faad4f4eb8a55b1
I20260807 22:50:43.532249 18063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:50:43.532370 18063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader afc993b5d4de4bb08faad4f4eb8a55b1)
I20260807 22:50:43.532476 18063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:50:43.535799 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.520310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:37461 (local address 127.0.0.1:58028)
0807 22:50:43.527398 (+ 7088us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.527416 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:43.527553 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:43.529773 (+ 2220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:43.529782 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:43.529825 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:43.530159 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.530176 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.531730 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.531737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.533785 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.533801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.535018 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.535023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.535194 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.535218 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:50:43.535326 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:50:43.535502 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6991,"thread_start_us":10103,"threads_started":1}
I20260807 22:50:43.535636 18063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } }
I20260807 22:50:43.536012 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.520707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.9.194.253:37461)
0807 22:50:43.529384 (+ 8677us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.529393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.529421 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:50:43.529492 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.529498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.529590 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.529718 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:43.530390 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.531520 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.533944 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.534778 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.534936 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.535064 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.535350 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.535501 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8513,"thread_start_us":8329,"threads_started":1}
I20260807 22:50:43.536767 18063 leader_election.cc:288] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 election: Requested vote from peers afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591), 0ca4789937cb494bb1782b208b1bc7e4 (127.9.194.252:34959)
I20260807 22:50:43.536857 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.536474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35516)
0807 22:50:43.536605 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.536693 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:50:43.537221 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72809db13eaa4248addf9720111c98c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ca4789937cb494bb1782b208b1bc7e4"
I20260807 22:50:43.537436 17493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:50:43.537544 17295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca4789937cb494bb1782b208b1bc7e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "72809db13eaa4248addf9720111c98c8" is_pre_election: true
I20260807 22:50:43.537912 17295 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ca4789937cb494bb1782b208b1bc7e4 in current term 2: Already voted for candidate 72809db13eaa4248addf9720111c98c8 in this term.
I20260807 22:50:43.538278 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.537791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35520)
0807 22:50:43.538030 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.538126 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260807 22:50:43.538563 17269 leader_election.cc:334] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591): Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
I20260807 22:50:43.538642 17463 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ca4789937cb494bb1782b208b1bc7e4; no voters: 72809db13eaa4248addf9720111c98c8, afc993b5d4de4bb08faad4f4eb8a55b1
I20260807 22:50:43.540194 17493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72809db13eaa4248addf9720111c98c8 in term 2.
I20260807 22:50:43.540369 18067 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:50:43.540635 17268 leader_election.cc:302] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ca4789937cb494bb1782b208b1bc7e4, 72809db13eaa4248addf9720111c98c8; no voters: afc993b5d4de4bb08faad4f4eb8a55b1
I20260807 22:50:43.540932 18063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:50:43.541468 18063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [term 2 LEADER]: Becoming Leader. State: Replica: 72809db13eaa4248addf9720111c98c8, State: Running, Role: LEADER
I20260807 22:50:43.541869 18063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } }
I20260807 22:50:43.543476 18073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72809db13eaa4248addf9720111c98c8. Latest consensus state: current_term: 2 leader_uuid: "72809db13eaa4248addf9720111c98c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:43.543946 18073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:43.544735 18075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:43.548734 18075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:43.549562 18075 catalog_manager.cc:1259] Loaded cluster ID: 607daf0a5a7b461e9a550e7203095e48
I20260807 22:50:43.549659 18075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:43.550827 18075 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:43.551710 18075 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Loaded TSK: 0
I20260807 22:50:43.552600 18075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:50:43.565376 17285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:43.572353 17285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:43.576737 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.576293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35530)
0807 22:50:43.576449 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.576566 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:50:43.576867 17493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [term 2 FOLLOWER]: Refusing update from remote peer 72809db13eaa4248addf9720111c98c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 22:50:43.577613 18073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 22:50:43.577780 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.577414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35534)
0807 22:50:43.577585 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.577655 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:50:43.577943 17269 consensus_peers.cc:487] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 -> Peer afc993b5d4de4bb08faad4f4eb8a55b1 (127.9.194.254:39591): Couldn't send request to peer afc993b5d4de4bb08faad4f4eb8a55b1. Status: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:50:43.580367 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72809db13eaa4248addf9720111c98c8. Latest consensus state: current_term: 2 leader_uuid: "72809db13eaa4248addf9720111c98c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:43.580632 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:43.582293 18063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "72809db13eaa4248addf9720111c98c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:43.582707 18063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:43.584064 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "72809db13eaa4248addf9720111c98c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "afc993b5d4de4bb08faad4f4eb8a55b1" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 39591 } } peers { permanent_uuid: "72809db13eaa4248addf9720111c98c8" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 37461 } attrs { promote: false } } peers { permanent_uuid: "0ca4789937cb494bb1782b208b1bc7e4" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 34959 } attrs { promote: false } } }
I20260807 22:50:43.584357 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca4789937cb494bb1782b208b1bc7e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:50:43.602320 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.590967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43887 (local address 127.0.0.1:51612)
0807 22:50:43.591138 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.591154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:43.591298 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:43.592901 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:43.592910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:43.592952 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:43.593261 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.593276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.595595 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.595606 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.598650 (+ 3044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.598675 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.601136 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.601146 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.601465 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.601527 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 22:50:43.601615 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 22:50:43.601864 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 22:50:43.605341 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.591092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.9.194.195:43887)
0807 22:50:43.592416 (+ 1324us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.592427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.592449 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:50:43.592530 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.592535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.592663 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.593448 (+ 785us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:43.593708 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.595343 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.598950 (+ 3607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.600876 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.604517 (+ 3641us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.604698 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.604769 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.605020 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":1361,"threads_started":1}
I20260807 22:50:43.606271 17910 tablet_service.cc:1467] Processing CreateTablet for tablet 5a0aa698df1b418eb2678cac9c35835a (DEFAULT_TABLE table=test-table [id=e11cdabcc47447a492e44e8dd36e5eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:43.606917 17910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a0aa698df1b418eb2678cac9c35835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:43.611466 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.592155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39959 (local address 127.0.0.1:51622)
0807 22:50:43.602573 (+ 10418us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.602592 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:43.602717 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:43.606060 (+ 3343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:43.606065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:43.606093 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:43.606293 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.606302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.607820 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.607825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.609345 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.609360 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.610525 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.610531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.610766 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.610791 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:50:43.610914 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:50:43.611068 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10330,"threads_started":1}
I20260807 22:50:43.611485 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.592309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.9.194.194:39959)
0807 22:50:43.605396 (+ 13087us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.605689 (+ 293us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.605711 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:50:43.605789 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.605794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.605875 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.606047 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:43.606462 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.607660 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.609512 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.610354 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.610516 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.610621 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.610946 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.611106 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12923,"thread_start_us":13263,"threads_started":1}
I20260807 22:50:43.616638 18087 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Bootstrap starting.
I20260807 22:50:43.619472 18087 tablet_bootstrap.cc:654] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:43.620042 17758 tablet_service.cc:1467] Processing CreateTablet for tablet 5a0aa698df1b418eb2678cac9c35835a (DEFAULT_TABLE table=test-table [id=e11cdabcc47447a492e44e8dd36e5eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:43.620483 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a0aa698df1b418eb2678cac9c35835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:43.621687 18087 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: No bootstrap required, opened a new log
I20260807 22:50:43.621902 18087 ts_tablet_manager.cc:1397] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260807 22:50:43.622785 18087 raft_consensus.cc:348] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.623032 18087 raft_consensus.cc:374] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:43.623123 18087 raft_consensus.cc:729] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517a04892fdf40c0a9a582ba9c273cd8, State: Initialized, Role: FOLLOWER
I20260807 22:50:43.623414 18087 consensus_queue.cc:260] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.626542 18087 ts_tablet_manager.cc:1428] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260807 22:50:43.626863 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.591369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:43437 (local address 127.0.0.1:38942)
0807 22:50:43.591727 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.591744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:43.591867 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:43.618885 (+ 27018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:43.618892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:43.618931 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:43.619325 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.619336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.621473 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.621480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.623762 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:43.623781 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.626054 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.626061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:43.626251 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.626288 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:50:43.626371 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:50:43.626536 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260807 22:50:43.627748 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.624644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:35931)
0807 22:50:43.627291 (+ 2647us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.627511 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2502,"thread_start_us":2319,"threads_started":1}
I20260807 22:50:43.627790 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.591723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.9.194.193:43437)
0807 22:50:43.618481 (+ 26758us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.618488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.618511 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:43.618597 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.618603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.618708 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.618837 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:43.619573 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.621257 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.624042 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.625758 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.625987 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.626168 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.627227 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.627409 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26615,"thread_start_us":27228,"threads_started":1}
I20260807 22:50:43.628407 17980 heartbeater.cc:503] Master 127.9.194.253:37461 was elected leader, sending a full tablet report...
I20260807 22:50:43.628686 17601 tablet_service.cc:1467] Processing CreateTablet for tablet 5a0aa698df1b418eb2678cac9c35835a (DEFAULT_TABLE table=test-table [id=e11cdabcc47447a492e44e8dd36e5eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:43.629123 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a0aa698df1b418eb2678cac9c35835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:43.629179 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.628454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:54351)
0807 22:50:43.628946 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.629035 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
I20260807 22:50:43.632038 18091 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Bootstrap starting.
I20260807 22:50:43.633492 18091 tablet_bootstrap.cc:654] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:43.634670 18092 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Bootstrap starting.
I20260807 22:50:43.635772 18091 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: No bootstrap required, opened a new log
I20260807 22:50:43.635984 18091 ts_tablet_manager.cc:1397] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 22:50:43.636296 18092 tablet_bootstrap.cc:654] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:43.636736 18091 raft_consensus.cc:348] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.636924 18091 raft_consensus.cc:374] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:43.637079 18091 raft_consensus.cc:729] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a18b33621054bb0b3e29e5f4b63b524, State: Initialized, Role: FOLLOWER
I20260807 22:50:43.637347 18091 consensus_queue.cc:260] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.638118 18091 ts_tablet_manager.cc:1428] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:50:43.639789 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.638454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:59369)
0807 22:50:43.639295 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.639502 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":567,"threads_started":1}
I20260807 22:50:43.639897 18092 tablet_bootstrap.cc:492] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: No bootstrap required, opened a new log
I20260807 22:50:43.640060 18092 ts_tablet_manager.cc:1397] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 22:50:43.641074 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.640625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:60583)
0807 22:50:43.640836 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.640911 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 22:50:43.641155 18092 raft_consensus.cc:348] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.641383 18092 raft_consensus.cc:374] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:43.641492 18092 raft_consensus.cc:729] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da55ca5ba27443c9abc68f0d333e203, State: Initialized, Role: FOLLOWER
I20260807 22:50:43.641752 18092 consensus_queue.cc:260] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.642453 18092 ts_tablet_manager.cc:1428] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:43.644198 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.643083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:49387)
0807 22:50:43.643675 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.643857 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":333,"threads_started":1}
I20260807 22:50:43.644407 17674 heartbeater.cc:503] Master 127.9.194.253:37461 was elected leader, sending a full tablet report...
I20260807 22:50:43.645478 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.644958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:50343)
0807 22:50:43.645198 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.645277 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260807 22:50:43.701812 18095 raft_consensus.cc:482] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:43.702085 18095 raft_consensus.cc:504] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.702826 18095 leader_election.cc:288] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887), 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:43.703374 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a0aa698df1b418eb2678cac9c35835a" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517a04892fdf40c0a9a582ba9c273cd8" is_pre_election: true
I20260807 22:50:43.703464 17623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a0aa698df1b418eb2678cac9c35835a" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da55ca5ba27443c9abc68f0d333e203" is_pre_election: true
I20260807 22:50:43.703697 17930 raft_consensus.cc:2454] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 0.
I20260807 22:50:43.703708 17623 raft_consensus.cc:2454] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 0.
I20260807 22:50:43.704211 17709 leader_election.cc:302] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18b33621054bb0b3e29e5f4b63b524, 517a04892fdf40c0a9a582ba9c273cd8; no voters:
I20260807 22:50:43.704481 18095 raft_consensus.cc:2781] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:43.704615 18095 raft_consensus.cc:482] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:43.704721 18095 raft_consensus.cc:3037] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:43.706866 18095 raft_consensus.cc:504] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.707363 18095 leader_election.cc:288] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 election: Requested vote from peers 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887), 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437)
I20260807 22:50:43.707904 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a0aa698df1b418eb2678cac9c35835a" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517a04892fdf40c0a9a582ba9c273cd8"
I20260807 22:50:43.708097 17930 raft_consensus.cc:3037] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:43.708052 17623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a0aa698df1b418eb2678cac9c35835a" candidate_uuid: "1a18b33621054bb0b3e29e5f4b63b524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da55ca5ba27443c9abc68f0d333e203"
I20260807 22:50:43.708266 17623 raft_consensus.cc:3037] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:43.710129 17930 raft_consensus.cc:2454] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 1.
I20260807 22:50:43.710337 17623 raft_consensus.cc:2454] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a18b33621054bb0b3e29e5f4b63b524 in term 1.
I20260807 22:50:43.710556 17709 leader_election.cc:302] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18b33621054bb0b3e29e5f4b63b524, 517a04892fdf40c0a9a582ba9c273cd8; no voters:
I20260807 22:50:43.710863 18095 raft_consensus.cc:2781] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:43.711290 18095 raft_consensus.cc:686] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 LEADER]: Becoming Leader. State: Replica: 1a18b33621054bb0b3e29e5f4b63b524, State: Running, Role: LEADER
I20260807 22:50:43.711627 18095 consensus_queue.cc:237] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } }
I20260807 22:50:43.711827 17828 heartbeater.cc:503] Master 127.9.194.253:37461 was elected leader, sending a full tablet report...
I20260807 22:50:43.712430 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.711967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:48181)
0807 22:50:43.712143 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.712260 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:50:43.714571 17285 catalog_manager.cc:5495] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "1a18b33621054bb0b3e29e5f4b63b524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a18b33621054bb0b3e29e5f4b63b524" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39959 } health_report { overall_health: HEALTHY } } }
I20260807 22:50:43.732100 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.727802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.9.194.194:39959)
0807 22:50:43.727962 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.727967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.727983 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:50:43.728143 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.728146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.728209 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.728281 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:43.728586 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.729264 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.730503 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.730908 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.730999 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.731344 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:43.731545 (+ 201us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:43.731618 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.731746 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.731805 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:50:43.734953 17623 raft_consensus.cc:1261] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Refusing update from remote peer 1a18b33621054bb0b3e29e5f4b63b524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:43.735036 17930 raft_consensus.cc:1261] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Refusing update from remote peer 1a18b33621054bb0b3e29e5f4b63b524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:43.735666 18099 consensus_queue.cc:1035] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2da55ca5ba27443c9abc68f0d333e203" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:43.736074 18095 consensus_queue.cc:1035] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "517a04892fdf40c0a9a582ba9c273cd8" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:43.739359 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.738721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:49765)
0807 22:50:43.738958 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.739124 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 22:50:43.739776 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.739318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:36537)
0807 22:50:43.739493 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.739597 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:50:43.745390 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.744720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.193:60039)
0807 22:50:43.745141 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.745234 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309}
I20260807 22:50:43.746296 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.745845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.194:57931)
0807 22:50:43.746037 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.746129 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 22:50:43.747803 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.746795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.9.194.195:45841)
0807 22:50:43.747481 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:43.747599 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507}
I20260807 22:50:43.751386 18104 mvcc.cc:205] Tried to move back new op lower bound from 7316041907132633088 to 7316041907045425152. Current Snapshot: MvccSnapshot[applied={T|T < 7316041907132633088}]
I20260807 22:50:43.752465 17282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:43.752723 17282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:43.760550 17282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72809db13eaa4248addf9720111c98c8: Sending DeleteTablet for 3 replicas of tablet 5a0aa698df1b418eb2678cac9c35835a
I20260807 22:50:43.761545 17758 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a0aa698df1b418eb2678cac9c35835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:43 UTC) from {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:51622
I20260807 22:50:43.761674 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a0aa698df1b418eb2678cac9c35835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:43 UTC) from {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:38942
I20260807 22:50:43.762172 17910 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a0aa698df1b418eb2678cac9c35835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:43 UTC) from {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:51612
I20260807 22:50:43.762194 17909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a0aa698df1b418eb2678cac9c35835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:43 UTC) from {username='oryx', principal='oryx/127.9.194.253@KRBTEST.COM'} at 127.0.0.1:51612
I20260807 22:50:43.763091 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 17457
I20260807 22:50:43.771941 18112 tablet_replica.cc:330] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: stopping tablet replica
I20260807 22:50:43.772198 18112 raft_consensus.cc:2229] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:43.772539 18112 raft_consensus.cc:2258] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:43.773938 18112 ts_tablet_manager.cc:1905] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:43.776935 18110 tablet_replica.cc:330] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: stopping tablet replica
I20260807 22:50:43.777143 18110 raft_consensus.cc:2229] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:43.777534 18110 raft_consensus.cc:2258] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:43.778426 18110 ts_tablet_manager.cc:1905] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:43.782637 18112 ts_tablet_manager.cc:1918] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:43.782799 18112 log.cc:1196] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/5a0aa698df1b418eb2678cac9c35835a
I20260807 22:50:43.783442 18112 ts_tablet_manager.cc:1939] T 5a0aa698df1b418eb2678cac9c35835a P 517a04892fdf40c0a9a582ba9c273cd8: Deleting consensus metadata
I20260807 22:50:43.783452 18110 ts_tablet_manager.cc:1918] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:43.783578 18110 log.cc:1196] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/5a0aa698df1b418eb2678cac9c35835a
I20260807 22:50:43.784068 18110 ts_tablet_manager.cc:1939] T 5a0aa698df1b418eb2678cac9c35835a P 1a18b33621054bb0b3e29e5f4b63b524: Deleting consensus metadata
I20260807 22:50:43.784960 17268 catalog_manager.cc:4841] TS 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887): tablet 5a0aa698df1b418eb2678cac9c35835a (table test-table [id=e11cdabcc47447a492e44e8dd36e5eca]) successfully deleted
I20260807 22:50:43.785224 17266 catalog_manager.cc:4841] TS 1a18b33621054bb0b3e29e5f4b63b524 (127.9.194.194:39959): tablet 5a0aa698df1b418eb2678cac9c35835a (table test-table [id=e11cdabcc47447a492e44e8dd36e5eca]) successfully deleted
W20260807 22:50:43.786185 17268 catalog_manager.cc:4808] TS 517a04892fdf40c0a9a582ba9c273cd8 (127.9.194.195:43887): delete failed for tablet 5a0aa698df1b418eb2678cac9c35835a because the tablet was not found. No further retry: Not found: Tablet not found: 5a0aa698df1b418eb2678cac9c35835a
I20260807 22:50:43.786267 18111 tablet_replica.cc:330] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: stopping tablet replica
I20260807 22:50:43.786484 18111 raft_consensus.cc:2229] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:43.786819 18111 raft_consensus.cc:2258] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:43.788951 18111 ts_tablet_manager.cc:1905] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:43.795018 18111 ts_tablet_manager.cc:1918] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:43.795153 18111 log.cc:1196] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/5a0aa698df1b418eb2678cac9c35835a
I20260807 22:50:43.795701 18111 ts_tablet_manager.cc:1939] T 5a0aa698df1b418eb2678cac9c35835a P 2da55ca5ba27443c9abc68f0d333e203: Deleting consensus metadata
I20260807 22:50:43.796813 17268 catalog_manager.cc:4841] TS 2da55ca5ba27443c9abc68f0d333e203 (127.9.194.193:43437): tablet 5a0aa698df1b418eb2678cac9c35835a (table test-table [id=e11cdabcc47447a492e44e8dd36e5eca]) successfully deleted
I20260807 22:50:43.799001 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.764715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.9.194.193:43437)
0807 22:50:43.764990 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.764999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.782400 (+ 17401us) server_negotiation.cc:407] Connection header received
0807 22:50:43.782698 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.782704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.782794 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.782862 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:43.785840 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.787013 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.788367 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.789002 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.789134 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.791947 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:43.791967 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:43.791988 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:43.792027 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:43.796086 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:43.796799 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:43.796806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:43.796809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:43.796968 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:43.797401 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:43.797409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:43.797416 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:43.797825 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:43.798127 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.798428 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.798675 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:50:43.799719 17666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:43.893975 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 17457
I20260807 22:50:43.914677 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 17685
I20260807 22:50:43.930254 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:43.915588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.9.194.194:39959)
0807 22:50:43.915745 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:43.915750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:43.915769 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:43.915915 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:43.915920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:43.916013 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:43.916121 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:43.917459 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.918636 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.920015 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:43.920672 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:43.920793 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:43.923412 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:43.923449 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:43.923470 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:43.923511 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:43.927463 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:43.928166 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:43.928176 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:43.928179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:43.928338 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:43.928666 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:43.928671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:43.928674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:43.929165 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:43.929456 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:43.929655 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 22:50:43.929872 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:50:43.931108 17821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:44.054400 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 17685
I20260807 22:50:44.079391 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 17840
I20260807 22:50:44.094019 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:44.080242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.9.194.195:43887)
0807 22:50:44.080410 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:44.080415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:44.080434 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:44.080696 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:44.080701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:44.080770 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:44.080871 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:44.082517 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:44.083382 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:44.084818 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:44.085278 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:44.085458 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:44.088235 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:44.088252 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:44.088269 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:44.088313 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:44.091127 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:44.091897 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:44.091905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:44.091907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:44.092047 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:44.092443 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:44.092454 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:44.092459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:44.092822 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:44.093130 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:44.093454 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 22:50:44.093696 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:50:44.094694 17972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:44.140612 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:44.139840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35544)
0807 22:50:44.140306 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:44.140427 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":258,"threads_started":1}
I20260807 22:50:44.141937 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:44.141595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:39591 (local address 127.0.0.1:35556)
0807 22:50:44.141738 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:44.141813 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:39591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 22:50:44.189750 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 17840
I20260807 22:50:44.209796 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 17130
I20260807 22:50:44.210928 17316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:44.336470 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 17130
W20260807 22:50:44.360378 17466 connection.cc:535] server connection from 127.0.0.1:41870 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:50:44.360807 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 17235
I20260807 22:50:44.361913 17515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:44.480052 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 17235
2026-08-07T22:50:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9730 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:44 dist-test-slave-5k54 krb5kdc[18125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:44 dist-test-slave-5k54 krb5kdc[18125](info): set up 2 sockets
Aug 07 22:50:44 dist-test-slave-5k54 krb5kdc[18125](info): commencing operation
krb5kdc: starting...
W20260807 22:50:45.629385 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:45 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab.
Entry for principal oryx/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab.
I20260807 22:50:45.883306 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43601
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab
--principal=oryx/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:38977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:46.210449 18140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:46.210721 18140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:46.210901 18140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:46.218034 18140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:46.218128 18140 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:50:46.218168 18140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:46.218230 18140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:46.218281 18140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:46.218334 18140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:46.228211 18140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:38977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43601
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.254.keytab
--principal=oryx/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:46.229578 18140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:46.230842 18140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:46.243974 18146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:46.243865 18147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:46.244061 18149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:46.245136 18140 server_base.cc:1034] running on GCE node
I20260807 22:50:46.245900 18140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:46.247249 18140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:46.248498 18140 hybrid_clock.cc:648] HybridClock initialized: now 1786143046248435 us; error 60 us; skew 500 ppm
I20260807 22:50:46.248837 18140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:46 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143046, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:46.252024 18140 init.cc:377] Logged in from keytab as oryx/127.9.194.254@KRBTEST.COM (short username oryx)
I20260807 22:50:46.253481 18140 webserver.cc:460] Webserver started at http://127.9.194.254:40337/ using document root <none> and password file <none>
I20260807 22:50:46.254199 18140 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:46.254329 18140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:46.254668 18140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:46.257599 18140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "11038a75989e4a179cbdd4a9ba69583c"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "1b813dd1cf3ab55afea15e32da0e5796"
server_key_iv: "320ca6640bb8eb842b2964f7f652e940"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.258406 18140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "11038a75989e4a179cbdd4a9ba69583c"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "1b813dd1cf3ab55afea15e32da0e5796"
server_key_iv: "320ca6640bb8eb842b2964f7f652e940"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.263533 18140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:50:46.266829 18157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:46.268143 18140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:46.268321 18140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "11038a75989e4a179cbdd4a9ba69583c"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "1b813dd1cf3ab55afea15e32da0e5796"
server_key_iv: "320ca6640bb8eb842b2964f7f652e940"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.268548 18140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:46.298305 18140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:46.302752 18140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:46.303032 18140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:46.326491 18140 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43601
I20260807 22:50:46.326511 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43601 every 8 connection(s)
I20260807 22:50:46.328464 18140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:46.333884 18222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:46.336593 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18140
I20260807 22:50:46.336840 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:46.337492 9995 external_mini_cluster.cc:1428] Setting key 31ab17fbe5109f70d48b7418f0247dbc
I20260807 22:50:46.351403 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Bootstrap starting.
I20260807 22:50:46.355144 18222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:46.356407 18222 log.cc:824] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:50:46 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:46.359730 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: No bootstrap required, opened a new log
I20260807 22:50:46.370289 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:46.339631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.9.194.254:43601)
0807 22:50:46.344064 (+ 4433us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:46.344080 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:46.344132 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:50:46.346618 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:46.346648 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:46.347216 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:46.347748 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:46.349372 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.351290 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.353385 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.354455 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.354634 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:46.359871 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:46.359883 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:46.359903 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:46.359953 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:46.364309 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.365400 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.365411 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.365424 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.365650 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.366136 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.366141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.366145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.366962 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:46.367266 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:46.368084 (+ 818us) server_negotiation.cc:299] Negotiation successful
0807 22:50:46.368625 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4149,"thread_start_us":3895,"threads_started":1}
I20260807 22:50:46.370314 18222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } }
I20260807 22:50:46.370827 18222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:46.370966 18222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11038a75989e4a179cbdd4a9ba69583c, State: Initialized, Role: FOLLOWER
I20260807 22:50:46.371830 18222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } }
I20260807 22:50:46.372161 18222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:46.372323 18222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:46.372540 18222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:46.375319 18222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } }
I20260807 22:50:46.375836 18222 leader_election.cc:302] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11038a75989e4a179cbdd4a9ba69583c; no voters:
I20260807 22:50:46.376513 18222 leader_election.cc:288] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:46.376673 18228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:46.377413 18228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [term 1 LEADER]: Becoming Leader. State: Replica: 11038a75989e4a179cbdd4a9ba69583c, State: Running, Role: LEADER
I20260807 22:50:46.378036 18228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } }
I20260807 22:50:46.378705 18222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:46.381785 18230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11038a75989e4a179cbdd4a9ba69583c. Latest consensus state: current_term: 1 leader_uuid: "11038a75989e4a179cbdd4a9ba69583c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } } }
I20260807 22:50:46.382174 18230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:46.383249 18235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:46.383363 18229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11038a75989e4a179cbdd4a9ba69583c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11038a75989e4a179cbdd4a9ba69583c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43601 } } }
I20260807 22:50:46.383682 18229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:46.389850 18235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:46.400511 18235 catalog_manager.cc:1347] Generated new cluster ID: 4a7a1845a1ec4aeebb555cbd42800720
I20260807 22:50:46.400624 18235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:46.417071 18235 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:46.418416 18235 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:46.423774 18235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Generated new TSK 0
I20260807 22:50:46.424657 18235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
I20260807 22:50:46.488336 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43601
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:38977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:46.815718 18250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:46.816002 18250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:46.816177 18250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:46.823565 18250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:46.823688 18250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:46.823837 18250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:50:46.823963 18250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:46.833766 18250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:38977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:43601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:46.835037 18250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:46.836400 18250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:46.849387 18255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:46.849474 18257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:46.849448 18259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:46.850572 18250 server_base.cc:1034] running on GCE node
I20260807 22:50:46.851204 18250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:46.852180 18250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:46.853389 18250 hybrid_clock.cc:648] HybridClock initialized: now 1786143046853308 us; error 52 us; skew 500 ppm
I20260807 22:50:46.853675 18250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:46 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143046, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:46.856817 18250 init.cc:377] Logged in from keytab as oryx/127.9.194.193@KRBTEST.COM (short username oryx)
I20260807 22:50:46.858265 18250 webserver.cc:460] Webserver started at http://127.9.194.193:45895/ using document root <none> and password file <none>
I20260807 22:50:46.858951 18250 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:46.859088 18250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:46.859406 18250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:46.862308 18250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "52d9b85a70b9f3427ef3fa7ea9830cf9"
server_key_iv: "5c3c378ea18914e9a36a2c4fd1fb4f8d"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.863093 18250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "52d9b85a70b9f3427ef3fa7ea9830cf9"
server_key_iv: "5c3c378ea18914e9a36a2c4fd1fb4f8d"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.868455 18250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 22:50:46.873503 18266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:46.875213 18250 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 22:50:46.875461 18250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
format_stamp: "Formatted at 2026-08-07 22:50:46 on dist-test-slave-5k54"
server_key: "52d9b85a70b9f3427ef3fa7ea9830cf9"
server_key_iv: "5c3c378ea18914e9a36a2c4fd1fb4f8d"
server_key_version: "encryptionkey@0"
I20260807 22:50:46.875775 18250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:46.896797 18250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:46.900828 18250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:46.901135 18250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:46.903899 18250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:46.904024 18250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:46.904166 18250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:46.904237 18250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:50:46 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143046, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:46.927079 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:46.904277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:32927 (local address 127.9.194.254:43601)
0807 22:50:46.904820 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:46.904829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:46.905967 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 22:50:46.908033 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:46.908039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:46.908133 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:46.908254 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:46.910347 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.911468 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.913976 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.914559 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.914706 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:46.919922 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:46.919931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:46.919939 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:46.920000 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:46.923472 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.924317 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.924323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.924326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.924458 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.924929 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.924935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.924938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.925213 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:46.925368 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:46.926172 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 22:50:46.926508 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":278,"threads_started":1}
I20260807 22:50:46.927645 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:46.905040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.193:32927)
0807 22:50:46.905808 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:46.905846 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:46.907756 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:46.908506 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:46.908520 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:46.909304 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:46.910150 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:46.910184 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.911638 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.911642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:46.913752 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:46.913763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.914828 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.914836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:46.915168 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:46.916062 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:46.916122 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:46.919685 (+ 3563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:46.923608 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:46.923617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:46.923635 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:46.924192 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:46.924568 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:46.924573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:46.924577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:46.924814 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:46.925343 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:46.925350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:46.925707 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 22:50:46.925985 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 22:50:46.926427 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":319,"threads_started":1}
I20260807 22:50:46.949108 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:46.934091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:53053 (local address 127.9.194.254:43601)
0807 22:50:46.934403 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:46.934411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:46.935272 (+ 861us) server_negotiation.cc:407] Connection header received
0807 22:50:46.935387 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:46.935395 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:46.935525 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:46.935636 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:46.937176 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.937879 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.939201 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.939672 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.939802 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:46.941956 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:46.941961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:46.941964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:46.941996 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:46.945983 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.946841 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.946845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.946847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.946970 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:46.947521 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:46.947528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:46.947533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:46.947818 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:46.947935 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:46.948475 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 22:50:46.948674 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 22:50:46.949476 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:46.934184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.193:53053)
0807 22:50:46.935159 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:46.935175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:46.935288 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:46.935799 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:46.935806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:46.936570 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:46.937016 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:46.937027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.938026 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.938031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:46.939061 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:46.939072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:46.939866 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:46.939874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:46.940047 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:46.940753 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:46.940773 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:46.941756 (+ 983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:46.946135 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:46.946145 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:46.946150 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:46.946655 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:46.947121 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:46.947133 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:46.947140 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:46.947345 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:46.947964 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:46.947969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:46.948202 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:50:46.948388 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 22:50:46.948799 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":534,"threads_started":1}
I20260807 22:50:46.958740 18250 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:46319
I20260807 22:50:46.958770 18392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:46319 every 8 connection(s)
I20260807 22:50:46.960060 18250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:46.962039 18393 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43601
I20260807 22:50:46.962232 18393 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:46.962826 18393 heartbeater.cc:511] Master 127.9.194.254:43601 requested a full tablet report, sending...
I20260807 22:50:46.963364 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18250
I20260807 22:50:46.963593 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:46.964406 9995 external_mini_cluster.cc:1428] Setting key 78f392705a93d96854d9d05483a926d3
I20260807 22:50:46.965202 18176 ts_manager.cc:194] Registered new tserver with Master: 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319)
I20260807 22:50:46.967067 18176 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:32927
WARNING: no policy specified for oryx/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
I20260807 22:50:47.020849 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43601
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:38977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:47.357719 18398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:47.358064 18398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:47.358235 18398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:47.365548 18398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:47.365710 18398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:47.365875 18398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:50:47.366024 18398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:47.376060 18398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:38977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:43601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:47.377390 18398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:47.378820 18398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:47.392469 18406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:47.392598 18403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:47.392513 18404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:47.393639 18398 server_base.cc:1034] running on GCE node
I20260807 22:50:47.394248 18398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:47.395117 18398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:47.396350 18398 hybrid_clock.cc:648] HybridClock initialized: now 1786143047396288 us; error 40 us; skew 500 ppm
I20260807 22:50:47.396656 18398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:47 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143047, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:47.399997 18398 init.cc:377] Logged in from keytab as oryx/127.9.194.194@KRBTEST.COM (short username oryx)
I20260807 22:50:47.401326 18398 webserver.cc:460] Webserver started at http://127.9.194.194:39895/ using document root <none> and password file <none>
I20260807 22:50:47.401996 18398 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:47.402108 18398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:47.402453 18398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:47.405398 18398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "05518494ef64b699adf541f95826cd05"
server_key_iv: "9dc6d9be76a75d48d47213fbd2644d06"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.406175 18398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "05518494ef64b699adf541f95826cd05"
server_key_iv: "9dc6d9be76a75d48d47213fbd2644d06"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.410929 18398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 22:50:47.414315 18413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:47.415628 18398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:47.415789 18398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "05518494ef64b699adf541f95826cd05"
server_key_iv: "9dc6d9be76a75d48d47213fbd2644d06"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.416008 18398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:47.438402 18398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:47.442487 18398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:47.442754 18398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:47.445523 18398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:47.445634 18398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:47.445783 18398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:47.445873 18398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:50:47 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143047, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:47.471009 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.446717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.194:36051)
0807 22:50:47.447679 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:47.447726 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:47.449594 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:47.450749 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:47.450765 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:47.451611 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:47.452603 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.452635 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.454070 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.454075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.456421 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.456433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.457530 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.457541 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.457898 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.458680 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:47.458725 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:47.462348 (+ 3623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:47.465849 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.465861 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.465883 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.466502 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.467055 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.467061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.467064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.467299 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.468055 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:47.468088 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:47.468468 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 22:50:47.468851 (+ 383us) client_negotiation.cc:240] Negotiation successful
0807 22:50:47.469427 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":407,"threads_started":1}
I20260807 22:50:47.472054 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.446080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:36051 (local address 127.9.194.254:43601)
0807 22:50:47.446285 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:47.446291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:47.447862 (+ 1571us) server_negotiation.cc:407] Connection header received
0807 22:50:47.449955 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:47.449964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:47.450055 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:47.450148 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:47.452816 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.453903 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.456574 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.457151 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.457305 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.462586 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:47.462591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:47.462595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:47.462629 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:47.465702 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.466683 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.466692 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.466699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.466877 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.467454 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.467460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.467465 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.467894 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:47.468091 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:47.471539 (+ 3448us) server_negotiation.cc:299] Negotiation successful
0807 22:50:47.471802 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:50:47.492131 18398 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:39167
I20260807 22:50:47.492138 18541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:39167 every 8 connection(s)
I20260807 22:50:47.493721 18398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:47.495580 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18398
I20260807 22:50:47.495780 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:47.495983 18542 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43601
I20260807 22:50:47.496268 18542 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:47.496348 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.480179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.194:44601)
0807 22:50:47.481053 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:47.481071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:47.481227 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:47.481605 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:47.481614 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:47.482354 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:47.482814 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.482831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.483911 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.483919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.486003 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.486015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.486831 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.486845 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.487087 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.487964 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:47.487993 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:47.489461 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:47.493178 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.493181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.493184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.493599 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.494021 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.494024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.494026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.494139 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.495194 (+ 1055us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:47.495199 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:47.495370 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 22:50:47.495543 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:50:47.495866 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":538,"threads_started":1}
I20260807 22:50:47.496366 9995 external_mini_cluster.cc:1428] Setting key 2f7baebec54e9cb387df6bd3720ce72f
I20260807 22:50:47.496503 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.479962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:44601 (local address 127.9.194.254:43601)
0807 22:50:47.480149 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:47.480155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:47.481158 (+ 1003us) server_negotiation.cc:407] Connection header received
0807 22:50:47.481350 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:47.481356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:47.481434 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:47.481529 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:47.482972 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.483750 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.486148 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.486568 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.486701 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.489735 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:47.489746 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:47.489749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:47.489779 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:47.493057 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.493723 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.493728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.493732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.493892 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.494284 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.494292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.494298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.495057 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:47.495220 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:47.495953 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 22:50:47.496172 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:50:47.497639 18542 heartbeater.cc:511] Master 127.9.194.254:43601 requested a full tablet report, sending...
I20260807 22:50:47.499083 18175 ts_manager.cc:194] Registered new tserver with Master: 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167)
I20260807 22:50:47.499990 18175 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:36051
WARNING: no policy specified for oryx/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
I20260807 22:50:47.554703 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43601
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:38977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:47.883206 18546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:47.883544 18546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:47.883662 18546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:47.890810 18546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:47.890906 18546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:47.891000 18546 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:50:47.891081 18546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:47.900682 18546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:38977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.194.254:43601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:47.901818 18546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:47.903098 18546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:47.916064 18551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:47.915978 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:47.916026 18554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:47.916960 18546 server_base.cc:1034] running on GCE node
I20260807 22:50:47.917548 18546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:47.918431 18546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:47.919685 18546 hybrid_clock.cc:648] HybridClock initialized: now 1786143047919593 us; error 50 us; skew 500 ppm
I20260807 22:50:47.919986 18546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:47 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143047, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:47.923092 18546 init.cc:377] Logged in from keytab as oryx/127.9.194.195@KRBTEST.COM (short username oryx)
I20260807 22:50:47.924535 18546 webserver.cc:460] Webserver started at http://127.9.194.195:41669/ using document root <none> and password file <none>
I20260807 22:50:47.925223 18546 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:47.925352 18546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:47.925666 18546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:47.928530 18546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "0d688e8af7784df988f86e9e595e2d4e"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "89d4b1ca422cf26bbb5ae3909a8bcdd8"
server_key_iv: "3f2e7924660db2f250b98d2be491b31e"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.929309 18546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "0d688e8af7784df988f86e9e595e2d4e"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "89d4b1ca422cf26bbb5ae3909a8bcdd8"
server_key_iv: "3f2e7924660db2f250b98d2be491b31e"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.934597 18546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:50:47.937922 18561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:47.939159 18546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:47.939332 18546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "0d688e8af7784df988f86e9e595e2d4e"
format_stamp: "Formatted at 2026-08-07 22:50:47 on dist-test-slave-5k54"
server_key: "89d4b1ca422cf26bbb5ae3909a8bcdd8"
server_key_iv: "3f2e7924660db2f250b98d2be491b31e"
server_key_version: "encryptionkey@0"
I20260807 22:50:47.939574 18546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:47.951629 18546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:47.955243 18546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:47.955583 18546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:47.958019 18546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:47.958123 18546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:47.958263 18546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:47.958333 18546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:47.969923 18393 heartbeater.cc:503] Master 127.9.194.254:43601 was elected leader, sending a full tablet report...
Aug 07 22:50:47 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143047, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:47.984144 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.959279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:44557 (local address 127.9.194.254:43601)
0807 22:50:47.959509 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:47.959515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:47.960672 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 22:50:47.963576 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:47.963583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:47.963676 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:47.963755 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:47.966547 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.967772 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.970269 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.971122 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.971278 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.977302 (+ 6024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:47.977310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:47.977316 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:47.977364 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:47.980919 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.981676 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.981683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.981686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.981839 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:47.982294 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:47.982300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:47.982303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:47.982588 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:47.982740 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:47.983531 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 22:50:47.983764 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:50:47.985531 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.959583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.195:44557)
0807 22:50:47.960504 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:47.960561 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:47.963200 (+ 2639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:47.964293 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:47.964305 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:47.965128 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:47.966347 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.966380 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.967964 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.967968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.970056 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.970070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.971690 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.971701 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.972062 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:47.973035 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:47.973080 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:47.977020 (+ 3940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:47.981074 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.981087 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.981104 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.981554 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.981952 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:47.981960 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:47.981964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:47.982175 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:47.982732 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:47.982740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:47.983093 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 22:50:47.983365 (+ 272us) client_negotiation.cc:240] Negotiation successful
0807 22:50:47.983951 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":381,"threads_started":1}
I20260807 22:50:48.009557 18546 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:33913
I20260807 22:50:48.009610 18686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:33913 every 8 connection(s)
I20260807 22:50:48.010010 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.992149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:33085 (local address 127.9.194.254:43601)
0807 22:50:47.992352 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:47.992360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:47.993525 (+ 1165us) server_negotiation.cc:407] Connection header received
0807 22:50:47.993775 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:47.993780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:47.993872 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:47.993988 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:47.995844 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.997122 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.999709 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:48.000161 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:48.000314 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:48.002782 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:48.002787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:48.002791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:48.002821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:48.006818 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:48.007684 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:48.007689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:48.007692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:48.007845 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:48.008238 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:48.008246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:48.008252 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:48.008692 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:48.008859 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:48.009409 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 22:50:48.009657 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:48.010150 18640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:47.992268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.9.194.195:33085)
0807 22:50:47.993416 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:47.993432 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:47.993616 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:47.994180 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:47.994187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:47.995026 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:47.995630 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.995648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:47.997569 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:47.997577 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:47.999564 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:47.999579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:48.000577 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:48.000583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:48.000707 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:48.001533 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:48.001550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:48.002586 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:48.006972 (+ 4386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:48.006981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:48.006986 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:48.007569 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:48.007983 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:48.007987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:48.007989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:48.008107 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:48.008829 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:48.008834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:48.009095 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 22:50:48.009295 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 22:50:48.009674 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":778,"threads_started":1}
I20260807 22:50:48.011106 18546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:48.013578 18687 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43601
I20260807 22:50:48.013837 18687 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:48.014482 18687 heartbeater.cc:511] Master 127.9.194.254:43601 requested a full tablet report, sending...
I20260807 22:50:48.015828 18175 ts_manager.cc:194] Registered new tserver with Master: 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:48.016640 18175 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:44557
I20260807 22:50:48.018257 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18546
I20260807 22:50:48.018431 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:48.018875 9995 external_mini_cluster.cc:1428] Setting key a3fe9be06806d8419170c9bab0a1e7f2
I20260807 22:50:48.021708 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:50:48 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:50:48 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:50:48 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:50:48.502506 18542 heartbeater.cc:503] Master 127.9.194.254:43601 was elected leader, sending a full tablet report...
I20260807 22:50:49.018915 18687 heartbeater.cc:503] Master 127.9.194.254:43601 was elected leader, sending a full tablet report...
W20260807 22:50:49.278322 18695 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.205s user 0.000s sys 0.018s
W20260807 22:50:49.279769 18275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:48.067553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.9.194.254:43601)
0807 22:50:48.067795 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:48.067803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:48.067936 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:50:48.068096 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:48.068100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:48.068165 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:48.068251 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:48.069625 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:48.070334 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:48.071683 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:48.072147 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:48.072247 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.279337 (+1207090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59760: BlockingRecv error: recv got EOF from 127.0.0.1:59760 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 22:50:49 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:49.302220 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.284974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.9.194.254:43601)
0807 22:50:49.285139 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.285143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.285463 (+ 320us) server_negotiation.cc:407] Connection header received
0807 22:50:49.285623 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.285627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.285704 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.285795 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:49.287065 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.287870 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.289145 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.289603 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.289760 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.297141 (+ 7381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:49.297146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:49.297150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:49.297185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:49.299699 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:49.300334 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:49.300341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:49.300344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:49.300468 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:49.300835 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:49.300841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:49.300845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:49.301131 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:49.301433 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.301701 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.301910 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:50:49.306288 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:49.313202 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:49.336618 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.325517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.0.0.1:50496)
0807 22:50:49.326424 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.326483 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.326689 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.328001 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.328008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.328038 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.328569 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.328588 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.330538 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.330546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.333104 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.333120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.335592 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.335598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.335782 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.335832 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.335954 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.336142 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":1,"thread_start_us":435,"threads_started":1}
I20260807 22:50:49.337754 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.326694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39167 (local address 127.0.0.1:37840)
0807 22:50:49.328736 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.328755 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.328863 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.329744 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.329748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.329772 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.330229 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.330241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.332509 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.332514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.334771 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.334788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.336929 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.336934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.337117 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.337153 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.337231 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.337363 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1890,"mutex_wait_us":90,"thread_start_us":1720,"threads_started":1}
I20260807 22:50:49.338202 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.326182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50496 (local address 127.9.194.193:46319)
0807 22:50:49.327327 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.327337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.327369 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:50:49.327508 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.327517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.327771 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.328003 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.328808 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.330373 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.333300 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.335161 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.335289 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.337425 (+ 2136us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.337516 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.337687 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":852,"threads_started":1}
I20260807 22:50:49.338443 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.327144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.9.194.194:39167)
0807 22:50:49.327715 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.327724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.328983 (+ 1259us) server_negotiation.cc:407] Connection header received
0807 22:50:49.329101 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.329107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.329380 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.329549 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.330463 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.332354 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.334960 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.336423 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.336565 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.336794 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.337975 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.338148 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
I20260807 22:50:49.344285 18465 tablet_service.cc:1467] Processing CreateTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d (DEFAULT_TABLE table=test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.345906 18465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 537414b3bbc74f8688ecf77af5e87f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.346130 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.330246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33913 (local address 127.0.0.1:47924)
0807 22:50:49.337120 (+ 6874us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.337139 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.337248 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.339559 (+ 2311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.339564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.339590 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.339849 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.339867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.341444 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.341455 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.343294 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.343308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.345245 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.345251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.345477 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.345546 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.345613 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.345750 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6779,"thread_start_us":9670,"threads_started":1}
I20260807 22:50:49.346011 18321 tablet_service.cc:1467] Processing CreateTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d (DEFAULT_TABLE table=test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.347244 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.328972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.9.194.195:33913)
0807 22:50:49.339054 (+ 10082us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.339062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.339085 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:49.339160 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.339167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.339336 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.339582 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.340070 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.341246 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.343562 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.345059 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.346529 (+ 1470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.346728 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.346795 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.346957 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9947,"thread_start_us":9827,"threads_started":1}
I20260807 22:50:49.347293 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 537414b3bbc74f8688ecf77af5e87f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.352826 18613 tablet_service.cc:1467] Processing CreateTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d (DEFAULT_TABLE table=test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.354205 18613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 537414b3bbc74f8688ecf77af5e87f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.359516 18712 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Bootstrap starting.
I20260807 22:50:49.361505 18713 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Bootstrap starting.
I20260807 22:50:49.363991 18712 tablet_bootstrap.cc:654] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.365347 18712 log.cc:824] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:49.365451 18713 tablet_bootstrap.cc:654] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.366580 18713 log.cc:824] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:49.368891 18712 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: No bootstrap required, opened a new log
I20260807 22:50:49.369216 18712 ts_tablet_manager.cc:1397] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 22:50:49.369652 18713 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: No bootstrap required, opened a new log
I20260807 22:50:49.369963 18713 ts_tablet_manager.cc:1397] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 22:50:49.371721 18716 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Bootstrap starting.
I20260807 22:50:49.374632 18716 tablet_bootstrap.cc:654] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.375821 18716 log.cc:824] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:49.380071 18712 raft_consensus.cc:348] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.380556 18712 raft_consensus.cc:374] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.380676 18712 raft_consensus.cc:729] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.380574 18713 raft_consensus.cc:348] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.381000 18713 raft_consensus.cc:374] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.381105 18713 raft_consensus.cc:729] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.381428 18712 consensus_queue.cc:260] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.381662 18713 consensus_queue.cc:260] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.383777 18712 ts_tablet_manager.cc:1428] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260807 22:50:49.387742 18716 tablet_bootstrap.cc:492] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: No bootstrap required, opened a new log
I20260807 22:50:49.388038 18716 ts_tablet_manager.cc:1397] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.010s
I20260807 22:50:49.388772 18713 ts_tablet_manager.cc:1428] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent starting tablet: real 0.019s user 0.007s sys 0.013s
I20260807 22:50:49.395550 18716 raft_consensus.cc:348] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.395908 18716 raft_consensus.cc:374] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.396027 18716 raft_consensus.cc:729] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.396602 18716 consensus_queue.cc:260] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.397912 18716 ts_tablet_manager.cc:1428] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
W20260807 22:50:49.463799 18394 tablet.cc:2369] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:50:49.496701 18543 tablet.cc:2369] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:50:49.513947 18688 tablet.cc:2369] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:50:49.570837 18718 raft_consensus.cc:482] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:49.571214 18718 raft_consensus.cc:504] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.573038 18718 leader_election.cc:288] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:49.579879 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.573438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:51423 (local address 127.9.194.194:39167)
0807 22:50:49.573632 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.573637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.574145 (+ 508us) server_negotiation.cc:407] Connection header received
0807 22:50:49.574500 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.574507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.574600 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.574677 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.575442 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.576264 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.578065 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.578804 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.579020 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.579099 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.579464 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.579609 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:50:49.579999 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.573314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39167 (local address 127.9.194.193:51423)
0807 22:50:49.574042 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.574062 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.574272 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.574877 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.574885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.574968 (+ 83us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.575268 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.575277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.576391 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.576398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.577903 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.577914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.579014 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.579021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.579192 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.579245 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.579390 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.579555 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"spinlock_wait_cycles":48896,"thread_start_us":353,"threads_started":1}
I20260807 22:50:49.580180 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.574237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:45033 (local address 127.9.194.195:33913)
0807 22:50:49.574445 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.574453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.574891 (+ 438us) server_negotiation.cc:407] Connection header received
0807 22:50:49.575133 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.575145 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.575217 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.575319 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.575940 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.576624 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.578342 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.579168 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.579408 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.579500 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.579807 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.579957 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:50:49.580272 18722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.574141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33913 (local address 127.9.194.193:45033)
0807 22:50:49.574761 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.574775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.574971 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.575332 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.575337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.575359 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.575769 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.575781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.576761 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.576770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.578173 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.578185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.579459 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.579465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.579614 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.579683 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.579765 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.579878 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260807 22:50:49.581322 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "537414b3bbc74f8688ecf77af5e87f4d" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" is_pre_election: true
I20260807 22:50:49.581321 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "537414b3bbc74f8688ecf77af5e87f4d" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e" is_pre_election: true
I20260807 22:50:49.581943 18485 raft_consensus.cc:2454] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 0.
I20260807 22:50:49.581954 18635 raft_consensus.cc:2454] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 0.
I20260807 22:50:49.582566 18271 leader_election.cc:302] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:49.582926 18718 raft_consensus.cc:2781] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:49.583076 18718 raft_consensus.cc:482] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:49.583185 18718 raft_consensus.cc:3037] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.585461 18718 raft_consensus.cc:504] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.586052 18718 leader_election.cc:288] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 election: Requested vote from peers 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:49.586478 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "537414b3bbc74f8688ecf77af5e87f4d" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
I20260807 22:50:49.586561 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "537414b3bbc74f8688ecf77af5e87f4d" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e"
I20260807 22:50:49.586706 18485 raft_consensus.cc:3037] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.586753 18635 raft_consensus.cc:3037] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.590054 18635 raft_consensus.cc:2454] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 1.
I20260807 22:50:49.590052 18485 raft_consensus.cc:2454] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 1.
I20260807 22:50:49.590441 18271 leader_election.cc:302] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:49.590775 18718 raft_consensus.cc:2781] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:49.591216 18718 raft_consensus.cc:686] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 LEADER]: Becoming Leader. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Running, Role: LEADER
I20260807 22:50:49.591739 18718 consensus_queue.cc:237] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.596195 18175 catalog_manager.cc:5495] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:49.651566 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.646276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.9.194.193:46319)
0807 22:50:49.646534 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.646540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.646557 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:50:49.646654 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.646658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.646749 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.646907 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:49.647294 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.648077 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.649403 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.649823 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.649959 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.650401 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:49.650938 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:49.651098 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.651158 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.651239 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:50:49.659694 18485 raft_consensus.cc:1261] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Refusing update from remote peer 84bcc1ee9a31424990c7f8a9b4b136f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:49.659693 18635 raft_consensus.cc:1261] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Refusing update from remote peer 84bcc1ee9a31424990c7f8a9b4b136f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:49.660722 18723 consensus_queue.cc:1035] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:49.661123 18718 consensus_queue.cc:1035] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:49.673782 18727 mvcc.cc:205] Tried to move back new op lower bound from 7316041931395960832 to 7316041931131674624. Current Snapshot: MvccSnapshot[applied={T|T < 7316041931395960832}]
I20260807 22:50:49.687681 18175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:49.687945 18175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:49.691787 18175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Sending DeleteTablet for 3 replicas of tablet 537414b3bbc74f8688ecf77af5e87f4d
I20260807 22:50:49.693240 18321 tablet_service.cc:1514] Processing DeleteTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:49 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:50496
I20260807 22:50:49.693233 18613 tablet_service.cc:1514] Processing DeleteTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:49 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47924
I20260807 22:50:49.693264 18465 tablet_service.cc:1514] Processing DeleteTablet for tablet 537414b3bbc74f8688ecf77af5e87f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:49 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37840
I20260807 22:50:49.697963 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:49.699910 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:49.703944 18735 tablet_replica.cc:330] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: stopping tablet replica
I20260807 22:50:49.704424 18735 raft_consensus.cc:2229] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:49.705232 18735 raft_consensus.cc:2258] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:49.709718 18735 ts_tablet_manager.cc:1905] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:49.711417 18612 tablet_service.cc:1467] Processing CreateTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 (DEFAULT_TABLE table=test-table [id=25ccbc75366647259935e39d1e952c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.711784 18321 tablet_service.cc:1467] Processing CreateTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 (DEFAULT_TABLE table=test-table [id=25ccbc75366647259935e39d1e952c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.712064 18612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0e0489a1e64f22b64d1535d8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.712364 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0e0489a1e64f22b64d1535d8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.714031 18464 tablet_service.cc:1467] Processing CreateTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 (DEFAULT_TABLE table=test-table [id=25ccbc75366647259935e39d1e952c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:49.714636 18464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0e0489a1e64f22b64d1535d8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.719230 18712 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Bootstrap starting.
I20260807 22:50:49.721015 18712 tablet_bootstrap.cc:654] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.722321 18738 tablet_replica.cc:330] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: stopping tablet replica
I20260807 22:50:49.722800 18738 raft_consensus.cc:2229] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:49.723661 18738 raft_consensus.cc:2258] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:49.725726 18738 ts_tablet_manager.cc:1905] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:49.727386 18716 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Bootstrap starting.
I20260807 22:50:49.729159 18716 tablet_bootstrap.cc:654] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.729995 18713 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Bootstrap starting.
I20260807 22:50:49.731982 18713 tablet_bootstrap.cc:654] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.740338 18712 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: No bootstrap required, opened a new log
I20260807 22:50:49.740523 18712 ts_tablet_manager.cc:1397] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.005s
I20260807 22:50:49.741511 18712 raft_consensus.cc:348] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.741832 18712 raft_consensus.cc:374] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.741981 18712 raft_consensus.cc:729] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.742368 18712 consensus_queue.cc:260] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.745867 18712 ts_tablet_manager.cc:1428] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260807 22:50:49.748158 18736 tablet_replica.cc:330] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: stopping tablet replica
I20260807 22:50:49.748373 18738 ts_tablet_manager.cc:1918] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:49.748551 18738 log.cc:1196] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/537414b3bbc74f8688ecf77af5e87f4d
I20260807 22:50:49.749002 18713 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: No bootstrap required, opened a new log
I20260807 22:50:49.749140 18738 ts_tablet_manager.cc:1939] T 537414b3bbc74f8688ecf77af5e87f4d P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting consensus metadata
I20260807 22:50:49.749171 18713 ts_tablet_manager.cc:1397] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.002s
I20260807 22:50:49.749619 18736 raft_consensus.cc:2229] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:49.750030 18713 raft_consensus.cc:348] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.750265 18713 raft_consensus.cc:374] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.750380 18713 raft_consensus.cc:729] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.750860 18736 raft_consensus.cc:2258] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:49.751159 18161 catalog_manager.cc:4841] TS 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167): tablet 537414b3bbc74f8688ecf77af5e87f4d (table test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]) successfully deleted
I20260807 22:50:49.750733 18713 consensus_queue.cc:260] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.752116 18716 tablet_bootstrap.cc:492] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: No bootstrap required, opened a new log
I20260807 22:50:49.752286 18716 ts_tablet_manager.cc:1397] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.008s
I20260807 22:50:49.753149 18716 raft_consensus.cc:348] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.753399 18716 raft_consensus.cc:374] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.753540 18716 raft_consensus.cc:729] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.753860 18716 consensus_queue.cc:260] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.754433 18716 ts_tablet_manager.cc:1428] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 22:50:49.755905 18713 ts_tablet_manager.cc:1428] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260807 22:50:49.757684 18735 ts_tablet_manager.cc:1918] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:49.757860 18736 ts_tablet_manager.cc:1905] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:49.757918 18735 log.cc:1196] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/537414b3bbc74f8688ecf77af5e87f4d
I20260807 22:50:49.758574 18735 ts_tablet_manager.cc:1939] T 537414b3bbc74f8688ecf77af5e87f4d P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting consensus metadata
I20260807 22:50:49.760453 18161 catalog_manager.cc:4841] TS 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319): tablet 537414b3bbc74f8688ecf77af5e87f4d (table test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]) successfully deleted
I20260807 22:50:49.763484 18736 ts_tablet_manager.cc:1918] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:49.763650 18736 log.cc:1196] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/537414b3bbc74f8688ecf77af5e87f4d
I20260807 22:50:49.764133 18736 ts_tablet_manager.cc:1939] T 537414b3bbc74f8688ecf77af5e87f4d P 0d688e8af7784df988f86e9e595e2d4e: Deleting consensus metadata
I20260807 22:50:49.765653 18161 catalog_manager.cc:4841] TS 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913): tablet 537414b3bbc74f8688ecf77af5e87f4d (table test-table [id=d26e56e8f55d4f3b8095b4e10dd42243]) successfully deleted
I20260807 22:50:49.812244 18720 raft_consensus.cc:482] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:49.812482 18720 raft_consensus.cc:504] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.814092 18720 leader_election.cc:288] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167)
I20260807 22:50:49.822407 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.814275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.9.194.195:55945)
0807 22:50:49.814887 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.814907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.815125 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.815872 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.815880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.815919 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.816388 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.816404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.818049 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.818060 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.820062 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.820075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.821438 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.821445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.821695 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.821722 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.821835 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.821981 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":36,"thread_start_us":308,"threads_started":1}
I20260807 22:50:49.823472 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.814453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:55945 (local address 127.9.194.193:46319)
0807 22:50:49.814655 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.814662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.815007 (+ 345us) server_negotiation.cc:407] Connection header received
0807 22:50:49.815342 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.815351 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.815498 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.815581 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.816588 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.817861 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.820219 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.821199 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.821401 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.821483 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.822978 (+ 1495us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.823135 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:50:49.823964 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.815245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:58421 (local address 127.9.194.194:39167)
0807 22:50:49.815415 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.815443 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.817122 (+ 1679us) server_negotiation.cc:407] Connection header received
0807 22:50:49.817313 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.817317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.817377 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.817476 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:49.818011 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.819136 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.821881 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.822976 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.823165 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.823240 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.823574 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.823700 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:50:49.824041 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.815030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39167 (local address 127.9.194.195:58421)
0807 22:50:49.817001 (+ 1971us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.817017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.817156 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.817501 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.817506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.817529 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:49.817839 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.817847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.819282 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.819293 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.821700 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.821718 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.823208 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.823214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.823378 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.823400 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.823532 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.823655 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1851,"mutex_wait_us":29,"thread_start_us":1670,"threads_started":1}
I20260807 22:50:49.824456 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0e0489a1e64f22b64d1535d8510ce5" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" is_pre_election: true
I20260807 22:50:49.825001 18346 raft_consensus.cc:2454] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 0.
I20260807 22:50:49.825757 18565 leader_election.cc:302] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:49.826220 18720 raft_consensus.cc:2781] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:49.826360 18720 raft_consensus.cc:482] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:49.826469 18720 raft_consensus.cc:3037] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.826745 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0e0489a1e64f22b64d1535d8510ce5" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" is_pre_election: true
I20260807 22:50:49.827008 18485 raft_consensus.cc:2454] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 0.
I20260807 22:50:49.829500 18720 raft_consensus.cc:504] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.830175 18720 leader_election.cc:288] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 election: Requested vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167)
I20260807 22:50:49.830610 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0e0489a1e64f22b64d1535d8510ce5" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
I20260807 22:50:49.830854 18346 raft_consensus.cc:3037] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.830783 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0e0489a1e64f22b64d1535d8510ce5" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
I20260807 22:50:49.831004 18485 raft_consensus.cc:3037] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:49.833328 18346 raft_consensus.cc:2454] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 1.
I20260807 22:50:49.833768 18565 leader_election.cc:302] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:49.834064 18720 raft_consensus.cc:2781] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:49.834074 18485 raft_consensus.cc:2454] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 1.
I20260807 22:50:49.834663 18720 raft_consensus.cc:686] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 1 LEADER]: Becoming Leader. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Running, Role: LEADER
I20260807 22:50:49.835060 18720 consensus_queue.cc:237] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.838855 18175 catalog_manager.cc:5495] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195). New cstate: current_term: 1 leader_uuid: "0d688e8af7784df988f86e9e595e2d4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } health_report { overall_health: HEALTHY } } }
Aug 07 22:50:49 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143046, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.254@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:49.916965 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.885877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43601 (local address 127.0.0.1:59786)
0807 22:50:49.901491 (+ 15614us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:49.901513 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:49.901669 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:49.902385 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:49.902396 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:49.903211 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:49.903589 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.903601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.904834 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.904841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.905928 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:49.905941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.906723 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.906729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:49.906955 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.907753 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:49.907772 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:49.910410 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:49.914071 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:49.914086 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:49.914095 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:49.914548 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:49.914967 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:49.914972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:49.914976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:49.915139 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:49.915771 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:49.915780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:49.915967 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 22:50:49.916138 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 22:50:49.916469 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15432,"thread_start_us":15257,"threads_started":1}
I20260807 22:50:49.917050 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:49.885858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.9.194.254:43601)
0807 22:50:49.902070 (+ 16212us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:49.902074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:49.902092 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:50:49.902132 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:49.902135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:49.902197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:49.902336 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:49.903789 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.904664 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.906087 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:49.906543 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:49.906700 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:49.910730 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:49.910738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:49.910743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:49.910784 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:49.913926 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:49.914685 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:49.914691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:49.914694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:49.914838 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:49.915262 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:49.915267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:49.915270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:49.915617 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:49.915824 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:49.916239 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 22:50:49.916621 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":16103,"thread_start_us":16629,"threads_started":1}
I20260807 22:50:49.922739 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:59786:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:50:49.923808 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:49.930647 18321 tablet_service.cc:1467] Processing CreateTablet for tablet 79648ad6874e4ff09fdc9501a4271783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fab21503a304990bd26c31f9101866c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:49.930771 18464 tablet_service.cc:1467] Processing CreateTablet for tablet 79648ad6874e4ff09fdc9501a4271783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fab21503a304990bd26c31f9101866c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:49.931039 18613 tablet_service.cc:1467] Processing CreateTablet for tablet 79648ad6874e4ff09fdc9501a4271783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fab21503a304990bd26c31f9101866c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:50:49.931286 18464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79648ad6874e4ff09fdc9501a4271783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.931315 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79648ad6874e4ff09fdc9501a4271783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.931727 18613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79648ad6874e4ff09fdc9501a4271783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:49.938750 18713 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: Bootstrap starting.
I20260807 22:50:49.939395 18716 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: Bootstrap starting.
I20260807 22:50:49.940263 18713 tablet_bootstrap.cc:654] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.941023 18716 tablet_bootstrap.cc:654] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.941217 18712 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: Bootstrap starting.
I20260807 22:50:49.942952 18712 tablet_bootstrap.cc:654] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:49.944192 18716 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: No bootstrap required, opened a new log
I20260807 22:50:49.944525 18716 ts_tablet_manager.cc:1397] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 22:50:49.944902 18713 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: No bootstrap required, opened a new log
I20260807 22:50:49.945087 18713 ts_tablet_manager.cc:1397] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260807 22:50:49.945813 18713 raft_consensus.cc:348] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.946080 18713 raft_consensus.cc:374] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.946163 18713 raft_consensus.cc:729] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.946419 18716 raft_consensus.cc:348] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.946475 18713 consensus_queue.cc:260] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.946825 18716 raft_consensus.cc:374] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.946965 18716 raft_consensus.cc:729] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.946951 18719 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:49.947299 18719 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:49.947520 18716 consensus_queue.cc:260] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.948117 18720 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:49.948455 18720 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:49.949155 18716 ts_tablet_manager.cc:1428] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:50:49.951017 18713 ts_tablet_manager.cc:1428] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260807 22:50:49.957754 18712 tablet_bootstrap.cc:492] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: No bootstrap required, opened a new log
I20260807 22:50:49.958015 18712 ts_tablet_manager.cc:1397] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.015s
I20260807 22:50:49.958837 18712 raft_consensus.cc:348] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.959095 18712 raft_consensus.cc:374] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:49.959179 18712 raft_consensus.cc:729] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Initialized, Role: FOLLOWER
I20260807 22:50:49.959511 18712 consensus_queue.cc:260] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:49.959932 18723 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:49.960250 18723 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:49.961146 18712 ts_tablet_manager.cc:1428] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:50:50.055229 18723 raft_consensus.cc:482] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:50.055543 18723 raft_consensus.cc:504] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.056291 18723 leader_election.cc:288] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:50.056959 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79648ad6874e4ff09fdc9501a4271783" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e" is_pre_election: true
I20260807 22:50:50.057003 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79648ad6874e4ff09fdc9501a4271783" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" is_pre_election: true
I20260807 22:50:50.057274 18635 raft_consensus.cc:2454] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 0.
I20260807 22:50:50.057332 18485 raft_consensus.cc:2454] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 0.
I20260807 22:50:50.057777 18271 leader_election.cc:302] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.058084 18723 raft_consensus.cc:2781] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:50.058199 18723 raft_consensus.cc:482] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:50.058337 18723 raft_consensus.cc:3037] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.060667 18723 raft_consensus.cc:504] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.061272 18723 leader_election.cc:288] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 election: Requested vote from peers 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:50.061625 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79648ad6874e4ff09fdc9501a4271783" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
I20260807 22:50:50.061815 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79648ad6874e4ff09fdc9501a4271783" candidate_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e"
I20260807 22:50:50.061889 18485 raft_consensus.cc:3037] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.062036 18635 raft_consensus.cc:3037] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.064117 18635 raft_consensus.cc:2454] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 1.
I20260807 22:50:50.064540 18271 leader_election.cc:302] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.064813 18723 raft_consensus.cc:2781] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:50.065032 18723 raft_consensus.cc:686] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 LEADER]: Becoming Leader. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Running, Role: LEADER
I20260807 22:50:50.065003 18485 raft_consensus.cc:2454] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bcc1ee9a31424990c7f8a9b4b136f8 in term 1.
I20260807 22:50:50.065294 18723 consensus_queue.cc:237] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.067209 18718 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: TxnStatusTablet state changed. Reason: New leader 84bcc1ee9a31424990c7f8a9b4b136f8. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.067554 18718 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:50.068686 18757 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:50:50.068946 18757 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:50:50.069463 18175 catalog_manager.cc:5495] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:50.111732 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.106486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.0.0.1:50510)
0807 22:50:50.106757 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.106778 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.106933 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.107363 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.107370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.107384 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:50.107575 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.107584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.108661 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.108667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.109674 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.109685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.110460 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.110466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.110630 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.110687 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.111104 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.111129 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.111228 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.111376 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:50:50.111728 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.106621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.9.194.193:46319)
0807 22:50:50.106826 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.106832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.106855 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:50.107138 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.107146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.107221 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.107321 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.107717 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.108506 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.109828 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.110307 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.110445 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.110878 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.110997 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.111102 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.111256 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.111346 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:50:50.115336 18635 raft_consensus.cc:1261] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Refusing update from remote peer 84bcc1ee9a31424990c7f8a9b4b136f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.115383 18485 raft_consensus.cc:1261] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Refusing update from remote peer 84bcc1ee9a31424990c7f8a9b4b136f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.115969 18718 consensus_queue.cc:1035] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.116261 18723 consensus_queue.cc:1035] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.120404 18719 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: TxnStatusTablet state changed. Reason: New leader 84bcc1ee9a31424990c7f8a9b4b136f8. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.120702 18719 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:50.122248 18718 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.122529 18718 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:50.122710 18726 mvcc.cc:205] Tried to move back new op lower bound from 7316041933266145280 to 7316041933069758464. Current Snapshot: MvccSnapshot[applied={T|T < 7316041933266145280}]
I20260807 22:50:50.123874 18720 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: TxnStatusTablet state changed. Reason: New leader 84bcc1ee9a31424990c7f8a9b4b136f8. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.124140 18720 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:50.125236 18719 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.125458 18719 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 4ef78aacc37946ea881c08d8ac15bd8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:50.125560 18728 mvcc.cc:205] Tried to move back new op lower bound from 7316041933266145280 to 7316041933069758464. Current Snapshot: MvccSnapshot[applied={T|T < 7316041933266145280 or (T in {7316041933266145280})}]
I20260807 22:50:50.127383 18723 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.128680 18723 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 84bcc1ee9a31424990c7f8a9b4b136f8: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:50:50.131040 18720 tablet_replica.cc:439] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } } }
I20260807 22:50:50.131338 18720 tablet_replica.cc:442] T 79648ad6874e4ff09fdc9501a4271783 P 0d688e8af7784df988f86e9e595e2d4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:50:50.138288 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.131576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.9.194.195:33913)
0807 22:50:50.132263 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.132269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.132292 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:50:50.132384 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.132390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.132488 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.132659 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.133109 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.134264 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.135822 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.136466 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.136588 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.137019 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.137685 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.137835 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.137879 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.137969 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":426,"threads_started":1}
I20260807 22:50:50.153769 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.148471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:33915 (local address 127.9.194.193:46319)
0807 22:50:50.148709 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.148715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.148894 (+ 179us) server_negotiation.cc:407] Connection header received
0807 22:50:50.149122 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.149127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.149199 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.149313 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.149796 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.150554 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.151973 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.152412 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.152571 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.152987 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.153126 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.153221 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.153356 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.153436 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:50:50.153862 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.148229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.9.194.195:33915)
0807 22:50:50.148799 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.148814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.148968 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.149324 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.149329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.149341 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:50.149651 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.149659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.150684 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.150689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.151838 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.151849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.152607 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.152615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.152757 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.152806 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.153224 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.153237 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.153334 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.153466 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":270,"threads_started":1}
I20260807 22:50:50.163369 18485 raft_consensus.cc:1261] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Refusing update from remote peer 0d688e8af7784df988f86e9e595e2d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.163794 18346 raft_consensus.cc:1261] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Refusing update from remote peer 0d688e8af7784df988f86e9e595e2d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.164628 18744 consensus_queue.cc:1035] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.165102 18720 consensus_queue.cc:1035] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.204244 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.197148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:42879 (local address 127.9.194.195:33913)
0807 22:50:50.197345 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.197352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.197829 (+ 477us) server_negotiation.cc:407] Connection header received
0807 22:50:50.198057 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.198063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.198138 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.198232 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.199283 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.200480 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.201927 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.202569 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.202696 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.203277 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.203443 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.203536 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.203675 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.203757 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:50:50.204257 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.197087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33913 (local address 127.9.194.193:42879)
0807 22:50:50.197718 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.197738 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.197905 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.198253 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.198258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.198269 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:50.198734 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.198749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.200605 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.200609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.201772 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.201784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.202783 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.202792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.203009 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.203092 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.203542 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.203554 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.203652 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.203803 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":321,"threads_started":1}
I20260807 22:50:50.258903 18176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:50.259186 18176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:50.262988 18176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Sending DeleteTablet for 3 replicas of tablet bc0e0489a1e64f22b64d1535d8510ce5
I20260807 22:50:50.264001 18464 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37840
I20260807 22:50:50.264447 18613 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47924
I20260807 22:50:50.264719 18773 tablet_replica.cc:330] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: stopping tablet replica
I20260807 22:50:50.264729 18736 tablet_replica.cc:330] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: stopping tablet replica
I20260807 22:50:50.264957 18736 raft_consensus.cc:2229] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:50.264998 18773 raft_consensus.cc:2229] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:50.265334 18773 raft_consensus.cc:2258] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:50.265507 18736 raft_consensus.cc:2258] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:50.267016 18773 ts_tablet_manager.cc:1905] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.267134 18736 ts_tablet_manager.cc:1905] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.272302 18321 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0e0489a1e64f22b64d1535d8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:50496
I20260807 22:50:50.274780 18773 ts_tablet_manager.cc:1918] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:50.274993 18773 log.cc:1196] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/bc0e0489a1e64f22b64d1535d8510ce5
I20260807 22:50:50.275694 18773 ts_tablet_manager.cc:1939] T bc0e0489a1e64f22b64d1535d8510ce5 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting consensus metadata
I20260807 22:50:50.276883 18161 catalog_manager.cc:4841] TS 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167): tablet bc0e0489a1e64f22b64d1535d8510ce5 (table test-table [id=25ccbc75366647259935e39d1e952c46]) successfully deleted
I20260807 22:50:50.285248 18736 ts_tablet_manager.cc:1918] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:50.289095 18736 log.cc:1196] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/bc0e0489a1e64f22b64d1535d8510ce5
I20260807 22:50:50.289825 18736 ts_tablet_manager.cc:1939] T bc0e0489a1e64f22b64d1535d8510ce5 P 0d688e8af7784df988f86e9e595e2d4e: Deleting consensus metadata
I20260807 22:50:50.291649 18161 catalog_manager.cc:4841] TS 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913): tablet bc0e0489a1e64f22b64d1535d8510ce5 (table test-table [id=25ccbc75366647259935e39d1e952c46]) successfully deleted
I20260807 22:50:50.294241 18774 tablet_replica.cc:330] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: stopping tablet replica
I20260807 22:50:50.294597 18774 raft_consensus.cc:2229] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:50.294945 18774 raft_consensus.cc:2258] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:50.296262 18774 ts_tablet_manager.cc:1905] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.303957 18774 ts_tablet_manager.cc:1918] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:50.304167 18774 log.cc:1196] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/bc0e0489a1e64f22b64d1535d8510ce5
I20260807 22:50:50.304749 18774 ts_tablet_manager.cc:1939] T bc0e0489a1e64f22b64d1535d8510ce5 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting consensus metadata
I20260807 22:50:50.306057 18161 catalog_manager.cc:4841] TS 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319): tablet bc0e0489a1e64f22b64d1535d8510ce5 (table test-table [id=25ccbc75366647259935e39d1e952c46]) successfully deleted
Aug 07 22:50:50 dist-test-slave-5k54 krb5kdc[18125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:50:50 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:50:50 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:50:50.375979 18751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:50.357830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.9.194.254:43601)
0807 22:50:50.358021 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.358027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.361877 (+ 3850us) server_negotiation.cc:407] Connection header received
0807 22:50:50.362092 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.362098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.362191 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.362314 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:50.364175 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.365383 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.367332 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.368055 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.368191 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.375686 (+ 7495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59792: BlockingRecv error: recv got EOF from 127.0.0.1:59792 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 22:50:50 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM
I20260807 22:50:50.399317 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.380426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.9.194.254:43601)
0807 22:50:50.380616 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.380622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.381239 (+ 617us) server_negotiation.cc:407] Connection header received
0807 22:50:50.381496 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.381502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.381585 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.381660 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:50.383466 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.384624 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.386591 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.387287 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.387412 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.392577 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:50.392587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:50.392594 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:50.392642 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:50.396313 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:50.397144 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:50.397150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:50.397154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:50.397321 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:50.397728 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:50.397734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:50.397738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:50.398109 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:50.398379 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.398685 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.398945 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:50:50.403565 18176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:50.404340 18176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:50.412858 18321 tablet_service.cc:1467] Processing CreateTablet for tablet 86a8131ed1af473a9545954eb875cb1e (DEFAULT_TABLE table=test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.412997 18464 tablet_service.cc:1467] Processing CreateTablet for tablet 86a8131ed1af473a9545954eb875cb1e (DEFAULT_TABLE table=test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.413331 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a8131ed1af473a9545954eb875cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.413426 18464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a8131ed1af473a9545954eb875cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.415831 18613 tablet_service.cc:1467] Processing CreateTablet for tablet 86a8131ed1af473a9545954eb875cb1e (DEFAULT_TABLE table=test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.416258 18613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a8131ed1af473a9545954eb875cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.420019 18713 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Bootstrap starting.
I20260807 22:50:50.421633 18713 tablet_bootstrap.cc:654] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.428949 18716 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Bootstrap starting.
I20260807 22:50:50.430760 18716 tablet_bootstrap.cc:654] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.432751 18713 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: No bootstrap required, opened a new log
I20260807 22:50:50.432902 18713 ts_tablet_manager.cc:1397] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260807 22:50:50.433697 18713 raft_consensus.cc:348] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.434233 18713 raft_consensus.cc:374] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.434345 18713 raft_consensus.cc:729] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.434607 18713 consensus_queue.cc:260] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.434821 18712 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Bootstrap starting.
I20260807 22:50:50.436300 18713 ts_tablet_manager.cc:1428] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 22:50:50.436734 18712 tablet_bootstrap.cc:654] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.439217 18712 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: No bootstrap required, opened a new log
I20260807 22:50:50.439370 18712 ts_tablet_manager.cc:1397] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 22:50:50.440203 18712 raft_consensus.cc:348] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.440534 18712 raft_consensus.cc:374] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.440629 18712 raft_consensus.cc:729] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.440904 18712 consensus_queue.cc:260] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.441954 18712 ts_tablet_manager.cc:1428] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:50:50.448653 18716 tablet_bootstrap.cc:492] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: No bootstrap required, opened a new log
I20260807 22:50:50.448817 18716 ts_tablet_manager.cc:1397] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.003s
I20260807 22:50:50.449642 18716 raft_consensus.cc:348] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.450323 18716 raft_consensus.cc:374] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.450475 18716 raft_consensus.cc:729] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.450723 18716 consensus_queue.cc:260] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.453843 18716 ts_tablet_manager.cc:1428] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 22:50:50.515272 18744 raft_consensus.cc:482] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:50.515565 18744 raft_consensus.cc:504] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.516270 18744 leader_election.cc:288] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167)
I20260807 22:50:50.517663 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a8131ed1af473a9545954eb875cb1e" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" is_pre_election: true
I20260807 22:50:50.517930 18346 raft_consensus.cc:2454] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 0.
I20260807 22:50:50.518302 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a8131ed1af473a9545954eb875cb1e" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" is_pre_election: true
I20260807 22:50:50.518535 18485 raft_consensus.cc:2454] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 0.
I20260807 22:50:50.519109 18565 leader_election.cc:302] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.519659 18744 raft_consensus.cc:2781] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:50.519759 18744 raft_consensus.cc:482] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:50.519860 18744 raft_consensus.cc:3037] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.522851 18744 raft_consensus.cc:504] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.524061 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a8131ed1af473a9545954eb875cb1e" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
I20260807 22:50:50.524296 18346 raft_consensus.cc:3037] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.527601 18346 raft_consensus.cc:2454] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 1.
I20260807 22:50:50.528080 18565 leader_election.cc:302] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d688e8af7784df988f86e9e595e2d4e, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.528424 18770 raft_consensus.cc:2781] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:50.528594 18770 raft_consensus.cc:686] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 1 LEADER]: Becoming Leader. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Running, Role: LEADER
I20260807 22:50:50.528944 18770 consensus_queue.cc:237] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.531517 18485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a8131ed1af473a9545954eb875cb1e" candidate_uuid: "0d688e8af7784df988f86e9e595e2d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef78aacc37946ea881c08d8ac15bd8a"
I20260807 22:50:50.531749 18485 raft_consensus.cc:3037] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.534791 18485 raft_consensus.cc:2454] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d688e8af7784df988f86e9e595e2d4e in term 1.
I20260807 22:50:50.535241 18744 leader_election.cc:288] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [CANDIDATE]: Term 1 election: Requested vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167)
I20260807 22:50:50.538820 18176 catalog_manager.cc:5495] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195). New cstate: current_term: 1 leader_uuid: "0d688e8af7784df988f86e9e595e2d4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } health_report { overall_health: HEALTHY } } }
I20260807 22:50:50.569029 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.562555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.9.194.195:33913)
0807 22:50:50.562859 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.562865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.562885 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:50:50.562998 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.563003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.563088 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.563192 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.563657 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.564799 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.566826 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.567568 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.567704 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.568189 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.568319 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.568411 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.568596 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.568682 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:50:50.572012 18485 raft_consensus.cc:1261] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Refusing update from remote peer 0d688e8af7784df988f86e9e595e2d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.573088 18744 consensus_queue.cc:1035] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 22:50:50.573457 18346 raft_consensus.cc:1261] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Refusing update from remote peer 0d688e8af7784df988f86e9e595e2d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.575541 18770 consensus_queue.cc:1035] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.588960 18175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:50.589258 18175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:50.593334 18175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Sending DeleteTablet for 3 replicas of tablet 86a8131ed1af473a9545954eb875cb1e
I20260807 22:50:50.594264 18321 tablet_service.cc:1514] Processing DeleteTablet for tablet 86a8131ed1af473a9545954eb875cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:50496
I20260807 22:50:50.594564 18774 tablet_replica.cc:330] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: stopping tablet replica
I20260807 22:50:50.594764 18774 raft_consensus.cc:2229] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:50.594941 18774 pending_rounds.cc:62] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Trying to abort 2 pending ops.
I20260807 22:50:50.595160 18774 pending_rounds.cc:68] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316041935136731136 op_type: WRITE_OP write_request { tablet_id: "86a8131ed1af473a9545954eb875cb1e" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\366\304\331\323\006\"8\n\ntest-admin\022*\n bf88b8eb104e4d33b8c0a2a55f3e3ce2\020\001\030\001 \001(\001" signature: "<redacted>""<9Eq\335\203\tg\262\nU\257\240\210\376\\\357\271:\370\256\222d\265\261\222\300\277\337\324\033\035\311\341\016g\262\003A\241\032w\342\r\245\211\340\230\324ud\014N\254\217\331\257\206\214\215\241{\016\017" signing_key_seq_num: 0 } } request_id { client_id: "ebd51a669b274923bc82aacc809fb889" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260807 22:50:50.596160 18774 pending_rounds.cc:68] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316041934969458688 op_type: NO_OP noop_request { }
I20260807 22:50:50.596338 18774 raft_consensus.cc:2866] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 22:50:50.596483 18774 raft_consensus.cc:2258] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:50:50.598012 18565 consensus_peers.cc:487] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e -> Peer 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319): Couldn't send request to peer 84bcc1ee9a31424990c7f8a9b4b136f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:50:50.598443 18464 tablet_service.cc:1514] Processing DeleteTablet for tablet 86a8131ed1af473a9545954eb875cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37840
I20260807 22:50:50.598764 18773 tablet_replica.cc:330] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: stopping tablet replica
I20260807 22:50:50.598995 18773 raft_consensus.cc:2229] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:50.599241 18773 raft_consensus.cc:2258] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:50.600021 18613 tablet_service.cc:1514] Processing DeleteTablet for tablet 86a8131ed1af473a9545954eb875cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:50 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47924
I20260807 22:50:50.600327 18736 tablet_replica.cc:330] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: stopping tablet replica
I20260807 22:50:50.600507 18736 raft_consensus.cc:2229] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:50.600927 18736 raft_consensus.cc:2258] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:50.602370 18736 ts_tablet_manager.cc:1905] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.603861 18774 ts_tablet_manager.cc:1905] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.605839 18773 ts_tablet_manager.cc:1905] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:50.611244 18774 ts_tablet_manager.cc:1918] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:50.611414 18774 log.cc:1196] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/86a8131ed1af473a9545954eb875cb1e
I20260807 22:50:50.612157 18774 ts_tablet_manager.cc:1939] T 86a8131ed1af473a9545954eb875cb1e P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting consensus metadata
I20260807 22:50:50.613720 18161 catalog_manager.cc:4841] TS 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319): tablet 86a8131ed1af473a9545954eb875cb1e (table test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]) successfully deleted
I20260807 22:50:50.618548 18736 ts_tablet_manager.cc:1918] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:50.618718 18736 log.cc:1196] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/86a8131ed1af473a9545954eb875cb1e
I20260807 22:50:50.619225 18736 ts_tablet_manager.cc:1939] T 86a8131ed1af473a9545954eb875cb1e P 0d688e8af7784df988f86e9e595e2d4e: Deleting consensus metadata
I20260807 22:50:50.620440 18174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:50:50.621212 18174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:50:50.621467 18773 ts_tablet_manager.cc:1918] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:50:50.621688 18773 log.cc:1196] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/86a8131ed1af473a9545954eb875cb1e
I20260807 22:50:50.622217 18773 ts_tablet_manager.cc:1939] T 86a8131ed1af473a9545954eb875cb1e P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting consensus metadata
I20260807 22:50:50.623230 18161 catalog_manager.cc:4841] TS 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913): tablet 86a8131ed1af473a9545954eb875cb1e (table test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]) successfully deleted
I20260807 22:50:50.624929 18161 catalog_manager.cc:4841] TS 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167): tablet 86a8131ed1af473a9545954eb875cb1e (table test-table [id=bf88b8eb104e4d33b8c0a2a55f3e3ce2]) successfully deleted
I20260807 22:50:50.630002 18613 tablet_service.cc:1467] Processing CreateTablet for tablet f9888011cee044559e122206707fde00 (DEFAULT_TABLE table=test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.630510 18613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9888011cee044559e122206707fde00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.636735 18464 tablet_service.cc:1467] Processing CreateTablet for tablet f9888011cee044559e122206707fde00 (DEFAULT_TABLE table=test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.637182 18464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9888011cee044559e122206707fde00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.637588 18716 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Bootstrap starting.
I20260807 22:50:50.635991 18321 tablet_service.cc:1467] Processing CreateTablet for tablet f9888011cee044559e122206707fde00 (DEFAULT_TABLE table=test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:50:50.638904 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9888011cee044559e122206707fde00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:50.639439 18716 tablet_bootstrap.cc:654] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.652292 18716 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: No bootstrap required, opened a new log
I20260807 22:50:50.652508 18716 ts_tablet_manager.cc:1397] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260807 22:50:50.653028 18713 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Bootstrap starting.
I20260807 22:50:50.653548 18716 raft_consensus.cc:348] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.653858 18716 raft_consensus.cc:374] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.653949 18716 raft_consensus.cc:729] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d688e8af7784df988f86e9e595e2d4e, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.654390 18716 consensus_queue.cc:260] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.654793 18713 tablet_bootstrap.cc:654] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.657954 18716 ts_tablet_manager.cc:1428] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260807 22:50:50.660145 18712 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Bootstrap starting.
I20260807 22:50:50.661727 18712 tablet_bootstrap.cc:654] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:50.663864 18712 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: No bootstrap required, opened a new log
I20260807 22:50:50.664018 18712 ts_tablet_manager.cc:1397] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 22:50:50.664746 18712 raft_consensus.cc:348] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.665057 18712 raft_consensus.cc:374] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.665143 18712 raft_consensus.cc:729] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcc1ee9a31424990c7f8a9b4b136f8, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.665423 18712 consensus_queue.cc:260] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.666069 18712 ts_tablet_manager.cc:1428] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:50:50.673028 18713 tablet_bootstrap.cc:492] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: No bootstrap required, opened a new log
I20260807 22:50:50.673199 18713 ts_tablet_manager.cc:1397] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.002s
I20260807 22:50:50.674068 18713 raft_consensus.cc:348] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.674582 18713 raft_consensus.cc:374] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:50.674672 18713 raft_consensus.cc:729] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Initialized, Role: FOLLOWER
I20260807 22:50:50.674965 18713 consensus_queue.cc:260] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.676249 18713 ts_tablet_manager.cc:1428] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 22:50:50.887681 18719 raft_consensus.cc:482] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:50:50.887916 18719 raft_consensus.cc:504] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.889529 18719 leader_election.cc:288] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:50.895591 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.889773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.9.194.194:48913)
0807 22:50:50.890396 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.890416 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.890548 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.891027 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.891032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.891059 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:50.891313 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.891322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.892349 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.892354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.893683 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.893695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.894778 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.894784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.894959 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.894983 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.895090 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.895220 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":303,"threads_started":1}
I20260807 22:50:50.895576 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.889887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:48913 (local address 127.9.194.193:46319)
0807 22:50:50.890376 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.890384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.890533 (+ 149us) server_negotiation.cc:407] Connection header received
0807 22:50:50.890778 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.890784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.890864 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.891003 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:50.891489 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.892190 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.893844 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.894595 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.894773 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.894851 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.895117 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.895261 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":252,"threads_started":1}
I20260807 22:50:50.896360 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.890659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:34341 (local address 127.9.194.195:33913)
0807 22:50:50.890855 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.890862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.891117 (+ 255us) server_negotiation.cc:407] Connection header received
0807 22:50:50.891327 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.891332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.891401 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.891534 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:50:50.892072 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.892751 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.894440 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.895244 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.895432 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.895538 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.896023 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.896170 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:50:50.896291 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9888011cee044559e122206707fde00" candidate_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" is_pre_election: true
I20260807 22:50:50.896447 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.890493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33913 (local address 127.9.194.194:34341)
0807 22:50:50.891009 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.891023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.891177 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.891557 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.891561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.891584 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:50:50.891911 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.891919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.892875 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.892883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.894289 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.894301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.895500 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.895507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.895806 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.895837 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.895960 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.896116 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":226,"threads_started":1}
I20260807 22:50:50.896572 18346 raft_consensus.cc:2454] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef78aacc37946ea881c08d8ac15bd8a in term 0.
I20260807 22:50:50.897122 18417 leader_election.cc:302] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef78aacc37946ea881c08d8ac15bd8a, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.897177 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9888011cee044559e122206707fde00" candidate_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e" is_pre_election: true
I20260807 22:50:50.897413 18635 raft_consensus.cc:2454] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef78aacc37946ea881c08d8ac15bd8a in term 0.
I20260807 22:50:50.897521 18719 raft_consensus.cc:2781] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:50:50.897645 18719 raft_consensus.cc:482] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:50:50.897750 18719 raft_consensus.cc:3037] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.900032 18719 raft_consensus.cc:504] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.900578 18719 leader_election.cc:288] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [CANDIDATE]: Term 1 election: Requested vote from peers 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319), 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913)
I20260807 22:50:50.901016 18346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9888011cee044559e122206707fde00" candidate_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8"
I20260807 22:50:50.901108 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9888011cee044559e122206707fde00" candidate_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d688e8af7784df988f86e9e595e2d4e"
I20260807 22:50:50.901208 18346 raft_consensus.cc:3037] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.901340 18635 raft_consensus.cc:3037] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:50.903159 18346 raft_consensus.cc:2454] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef78aacc37946ea881c08d8ac15bd8a in term 1.
I20260807 22:50:50.903311 18635 raft_consensus.cc:2454] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef78aacc37946ea881c08d8ac15bd8a in term 1.
I20260807 22:50:50.903563 18417 leader_election.cc:302] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef78aacc37946ea881c08d8ac15bd8a, 84bcc1ee9a31424990c7f8a9b4b136f8; no voters:
I20260807 22:50:50.903872 18719 raft_consensus.cc:2781] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:50.904270 18719 raft_consensus.cc:686] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 LEADER]: Becoming Leader. State: Replica: 4ef78aacc37946ea881c08d8ac15bd8a, State: Running, Role: LEADER
I20260807 22:50:50.904639 18719 consensus_queue.cc:237] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } }
I20260807 22:50:50.908205 18174 catalog_manager.cc:5495] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194). New cstate: current_term: 1 leader_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef78aacc37946ea881c08d8ac15bd8a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 } health_report { overall_health: UNKNOWN } } }
I20260807 22:50:50.956806 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.951308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.9.194.194:39167)
0807 22:50:50.951912 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.951917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.951936 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:50:50.952017 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.952020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.952100 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.952237 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.952574 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.953337 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.954569 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.954968 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.955060 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.955505 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.956169 (+ 664us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.956276 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.956376 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.956437 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":376,"threads_started":1}
I20260807 22:50:50.972437 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.966523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:48483 (local address 127.9.194.193:46319)
0807 22:50:50.966763 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:50.966770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:50.967090 (+ 320us) server_negotiation.cc:407] Connection header received
0807 22:50:50.967327 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:50.967333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:50.967414 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:50.967536 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:50.967937 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.969005 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.970313 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.971002 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.971114 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.971735 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.971860 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.971944 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:50.972107 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 22:50:50.972183 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:50:50.972638 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:50.966405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46319 (local address 127.9.194.194:48483)
0807 22:50:50.966991 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:50.967006 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:50.967163 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:50.967553 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:50.967556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:50.967566 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:50.967804 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.967810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.969134 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.969138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.970186 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:50.970198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:50.971213 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:50.971219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:50.971489 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:50.971555 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:50.971953 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:50.971991 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:50:50.972072 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:50:50.972169 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":265,"threads_started":1}
I20260807 22:50:50.981230 18635 raft_consensus.cc:1261] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Refusing update from remote peer 4ef78aacc37946ea881c08d8ac15bd8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.981527 18346 raft_consensus.cc:1261] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Refusing update from remote peer 4ef78aacc37946ea881c08d8ac15bd8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:50:50.981824 18719 consensus_queue.cc:1035] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d688e8af7784df988f86e9e595e2d4e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:50.982632 18719 consensus_queue.cc:1035] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bcc1ee9a31424990c7f8a9b4b136f8" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:50:51.011200 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:51.005556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37717 (local address 127.9.194.194:39167)
0807 22:50:51.005709 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:51.005714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:51.006384 (+ 670us) server_negotiation.cc:407] Connection header received
0807 22:50:51.006540 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:51.006544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:51.006603 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:51.006674 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:50:51.007075 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.007955 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.009456 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.010082 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.010197 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:51.010547 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:50:51.010658 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:50:51.010741 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:51.010841 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 22:50:51.010916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:50:51.011559 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:51.005480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:39167 (local address 127.9.194.193:37717)
0807 22:50:51.006295 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:51.006310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:51.006428 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:51.006719 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:51.006722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:51.006731 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:50:51.006957 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:51.006964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.008104 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.008108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:51.009308 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:51.009331 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.010227 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.010231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:51.010358 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:51.010393 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:50:51.010753 (+ 360us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:50:51.010761 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:50:51.010818 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:50:51.010916 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":543,"threads_started":1}
I20260807 22:50:51.043982 18176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:51.044229 18176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:50:51.047636 18176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11038a75989e4a179cbdd4a9ba69583c: Sending DeleteTablet for 3 replicas of tablet f9888011cee044559e122206707fde00
I20260807 22:50:51.048573 18464 tablet_service.cc:1514] Processing DeleteTablet for tablet f9888011cee044559e122206707fde00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:51 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37840
I20260807 22:50:51.048878 18773 tablet_replica.cc:330] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: stopping tablet replica
I20260807 22:50:51.049063 18773 raft_consensus.cc:2229] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:50:51.049425 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18250
I20260807 22:50:51.049566 18773 raft_consensus.cc:2258] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:51.050107 18613 tablet_service.cc:1514] Processing DeleteTablet for tablet f9888011cee044559e122206707fde00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:51 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47924
I20260807 22:50:51.050295 18321 tablet_service.cc:1514] Processing DeleteTablet for tablet f9888011cee044559e122206707fde00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:50:51 UTC) from {username='oryx', principal='oryx/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:50496
I20260807 22:50:51.050609 18774 tablet_replica.cc:330] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: stopping tablet replica
I20260807 22:50:51.050653 18736 tablet_replica.cc:330] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: stopping tablet replica
I20260807 22:50:51.050798 18774 raft_consensus.cc:2229] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:51.050868 18736 raft_consensus.cc:2229] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:50:51.051041 18774 raft_consensus.cc:2258] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:51.051076 18736 raft_consensus.cc:2258] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:50:51.052496 18773 ts_tablet_manager.cc:1905] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:51.052592 18774 ts_tablet_manager.cc:1905] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:51.059435 18774 ts_tablet_manager.cc:1918] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:51.059651 18774 log.cc:1196] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/f9888011cee044559e122206707fde00
I20260807 22:50:51.060178 18774 ts_tablet_manager.cc:1939] T f9888011cee044559e122206707fde00 P 84bcc1ee9a31424990c7f8a9b4b136f8: Deleting consensus metadata
I20260807 22:50:51.061344 18161 catalog_manager.cc:4841] TS 84bcc1ee9a31424990c7f8a9b4b136f8 (127.9.194.193:46319): tablet f9888011cee044559e122206707fde00 (table test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]) successfully deleted
I20260807 22:50:51.062002 18773 ts_tablet_manager.cc:1918] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:51.062178 18773 log.cc:1196] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/f9888011cee044559e122206707fde00
I20260807 22:50:51.062758 18773 ts_tablet_manager.cc:1939] T f9888011cee044559e122206707fde00 P 4ef78aacc37946ea881c08d8ac15bd8a: Deleting consensus metadata
I20260807 22:50:51.064493 18736 ts_tablet_manager.cc:1905] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:50:51.073182 18736 ts_tablet_manager.cc:1918] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:50:51.073355 18736 log.cc:1196] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/f9888011cee044559e122206707fde00
I20260807 22:50:51.073813 18736 ts_tablet_manager.cc:1939] T f9888011cee044559e122206707fde00 P 0d688e8af7784df988f86e9e595e2d4e: Deleting consensus metadata
I20260807 22:50:51.075323 18161 catalog_manager.cc:4841] TS 4ef78aacc37946ea881c08d8ac15bd8a (127.9.194.194:39167): tablet f9888011cee044559e122206707fde00 (table test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]) successfully deleted
I20260807 22:50:51.075964 18161 catalog_manager.cc:4841] TS 0d688e8af7784df988f86e9e595e2d4e (127.9.194.195:33913): tablet f9888011cee044559e122206707fde00 (table test-table [id=0bbf85056b9a4fcca01bf29c3c0d5f08]) successfully deleted
Aug 07 22:50:51 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.193@KRBTEST.COM
I20260807 22:50:51.088109 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:51.051153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.9.194.193:46319)
0807 22:50:51.051342 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:51.051351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:51.072037 (+ 20686us) server_negotiation.cc:407] Connection header received
0807 22:50:51.072202 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:51.072207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:51.072291 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:51.072405 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:51.074443 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.075540 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.076968 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.077390 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.077526 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:51.081966 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:51.081994 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:51.082010 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:51.082045 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:51.085144 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.085875 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.085883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.085885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.086020 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.086403 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.086409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.086412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.086760 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:51.087088 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:51.087435 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 22:50:51.087682 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:50:51.088902 18388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:51.254547 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18250
I20260807 22:50:51.294337 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18398
Aug 07 22:50:51 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.194@KRBTEST.COM
I20260807 22:50:51.311777 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:51.295208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.9.194.194:39167)
0807 22:50:51.295413 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:51.295417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:51.295538 (+ 121us) server_negotiation.cc:407] Connection header received
0807 22:50:51.295715 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:51.295719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:51.295784 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:51.295880 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:51.297550 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.298309 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.300163 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.300607 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.300700 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:51.305784 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:51.305798 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:51.305812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:51.305852 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:51.308973 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.309648 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.309654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.309657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.309802 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.310210 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.310214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.310216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.310567 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:51.310943 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:51.311151 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 22:50:51.311390 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:50:51.312417 18537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:51.469575 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18398
I20260807 22:50:51.502943 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18546
Aug 07 22:50:51 dist-test-slave-5k54 krb5kdc[18125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.194.195@KRBTEST.COM
I20260807 22:50:51.527513 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:51.504210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.9.194.195:33913)
0807 22:50:51.505029 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:51.505034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:51.505052 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:50:51.505127 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:51.505132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:51.505245 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:51.505428 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:51.507792 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.508914 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.510890 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:51.511616 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:51.511810 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:51.519071 (+ 7261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:51.519096 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:51.519125 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:51.519204 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:51.524269 (+ 5065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.525219 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.525227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.525229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.525485 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:51.525969 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:51.525972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:51.525982 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:51.526424 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:51.526640 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:51.526886 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 22:50:51.527072 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":472,"threads_started":1}
I20260807 22:50:51.528544 18682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:51.708077 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18546
I20260807 22:50:51.744213 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18140
I20260807 22:50:51.745659 18217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:51.898474 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18140
2026-08-07T22:50:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7431 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:52 dist-test-slave-5k54 krb5kdc[18829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:52 dist-test-slave-5k54 krb5kdc[18829](info): set up 2 sockets
Aug 07 22:50:52 dist-test-slave-5k54 krb5kdc[18829](info): commencing operation
krb5kdc: starting...
W20260807 22:50:53.502745 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.539s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:53 dist-test-slave-5k54 krb5kdc[18829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:53.895864 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35043
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:35043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:54.243903 18844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:54.244259 18844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:54.244421 18844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:54.251792 18844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:54.251950 18844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:54.252054 18844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:54.252157 18844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:54.252236 18844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:54.261940 18844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:35043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:54.263276 18844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:54.264632 18844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:54.277508 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:54.277585 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:54.277585 18852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:54.278479 18844 server_base.cc:1034] running on GCE node
I20260807 22:50:54.279196 18844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:54.280599 18844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:54.281803 18844 hybrid_clock.cc:648] HybridClock initialized: now 1786143054281762 us; error 31 us; skew 500 ppm
I20260807 22:50:54.282171 18844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T22:50:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2429 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:50:54 dist-test-slave-5k54 krb5kdc[18858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:50:54 dist-test-slave-5k54 krb5kdc[18858](info): set up 2 sockets
Aug 07 22:50:54 dist-test-slave-5k54 krb5kdc[18858](info): commencing operation
krb5kdc: starting...
W20260807 22:50:55.577302 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:50:55 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:50:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:50:55.847242 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43971
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:56.180219 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:56.180501 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:56.180591 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:56.187824 18871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:56.187937 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:56.187995 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:56.188052 18871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:56.188097 18871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:56.197815 18871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:56.199026 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:56.200373 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:56.212486 18876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:56.212510 18877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:56.212544 18879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:56.213737 18871 server_base.cc:1034] running on GCE node
I20260807 22:50:56.214495 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:56.215854 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:56.217067 18871 hybrid_clock.cc:648] HybridClock initialized: now 1786143056217029 us; error 31 us; skew 500 ppm
I20260807 22:50:56.217362 18871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:56 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:56.220526 18871 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:56.222096 18871 webserver.cc:460] Webserver started at http://127.9.194.254:38091/ using document root <none> and password file <none>
I20260807 22:50:56.222747 18871 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:56.222858 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:56.223122 18871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:56.226183 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "bb9b223e26c3388c97bd3d4dd0acc873"
server_key_iv: "259bd8ba73bf996fa2de4490a0c8f90d"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.226989 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "bb9b223e26c3388c97bd3d4dd0acc873"
server_key_iv: "259bd8ba73bf996fa2de4490a0c8f90d"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.232981 18871 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260807 22:50:56.236779 18886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:56.238260 18871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:50:56.238471 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "bb9b223e26c3388c97bd3d4dd0acc873"
server_key_iv: "259bd8ba73bf996fa2de4490a0c8f90d"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.238690 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:56.251868 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:56.255908 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:56.256199 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:56.276015 18871 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43971
I20260807 22:50:56.276037 18938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43971 every 8 connection(s)
I20260807 22:50:56.277392 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:56.281059 18939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:50:56.282311 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18871
I20260807 22:50:56.282533 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:50:56.283205 9995 external_mini_cluster.cc:1428] Setting key 91b108140ce912a6bd971767fa86e259
I20260807 22:50:56.295329 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Bootstrap starting.
Aug 07 22:50:56 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:56.298395 18939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Neither blocks nor log segments found. Creating new log.
I20260807 22:50:56.299857 18939 log.cc:824] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:56.303215 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: No bootstrap required, opened a new log
I20260807 22:50:56.307369 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:56.285260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.9.194.254:43971)
0807 22:50:56.286102 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:56.286126 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:56.286176 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:50:56.287926 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:56.287957 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:56.288455 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:56.288890 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:56.290388 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.292488 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.293739 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.294738 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.294921 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:56.298763 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:56.298776 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:56.298798 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:56.298852 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:56.302850 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:56.303622 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:56.303632 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:56.303643 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:56.303868 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:56.304225 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:56.304230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:56.304233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:56.304922 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:56.305230 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:56.305550 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 22:50:56.306010 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":358,"threads_started":1}
I20260807 22:50:56.312790 18939 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:56.313269 18939 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:50:56.313408 18939 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35b3e72b6e49d6af7b72ef6f8bc1fe, State: Initialized, Role: FOLLOWER
I20260807 22:50:56.313961 18939 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:56.314205 18939 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:56.314352 18939 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:56.314538 18939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:50:56.316762 18939 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:56.317235 18939 leader_election.cc:302] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc35b3e72b6e49d6af7b72ef6f8bc1fe; no voters:
I20260807 22:50:56.317799 18939 leader_election.cc:288] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:50:56.317974 18944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:50:56.318655 18944 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 LEADER]: Becoming Leader. State: Replica: cc35b3e72b6e49d6af7b72ef6f8bc1fe, State: Running, Role: LEADER
I20260807 22:50:56.319289 18944 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:56.320119 18939 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:56.322994 18946 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc35b3e72b6e49d6af7b72ef6f8bc1fe. Latest consensus state: current_term: 1 leader_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } } }
I20260807 22:50:56.322881 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } } }
I20260807 22:50:56.323279 18946 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:56.323325 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:56.325953 18953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:56.330232 18953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:56.338559 18953 catalog_manager.cc:1347] Generated new cluster ID: 7455dfc59ff24dbaac1f4e1ec9097eab
I20260807 22:50:56.338670 18953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:56.357260 18953 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:50:56.358588 18953 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:56.364837 18953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Generated new TSK 0
I20260807 22:50:56.365809 18953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:50:56.451159 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:56.789973 18968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:56.790458 18968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:56.790673 18968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:56.800731 18968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:56.800886 18968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:56.801064 18968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:56.811993 18968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:56.813321 18968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:56.814671 18968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:56.829783 18974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:56.829944 18975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:56.830214 18977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:56.831087 18968 server_base.cc:1034] running on GCE node
I20260807 22:50:56.832432 18968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:56.833968 18968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:56.835281 18968 hybrid_clock.cc:648] HybridClock initialized: now 1786143056835202 us; error 57 us; skew 500 ppm
I20260807 22:50:56.835876 18968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:56 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:56.840538 18968 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:50:56.842777 18968 webserver.cc:460] Webserver started at http://127.9.194.193:33859/ using document root <none> and password file <none>
I20260807 22:50:56.843928 18968 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:56.844095 18968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:56.844537 18968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:56.849915 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "c243db2271dd4c9aa2004b08ecb7a561"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "ad7b2d2667d1990c9837232161ced559"
server_key_iv: "4c88d4285143d9ef65d14c4446eed5d6"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.851264 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "c243db2271dd4c9aa2004b08ecb7a561"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "ad7b2d2667d1990c9837232161ced559"
server_key_iv: "4c88d4285143d9ef65d14c4446eed5d6"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.859354 18968 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260807 22:50:56.865000 18984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:56.866794 18968 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 22:50:56.867030 18968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "c243db2271dd4c9aa2004b08ecb7a561"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "ad7b2d2667d1990c9837232161ced559"
server_key_iv: "4c88d4285143d9ef65d14c4446eed5d6"
server_key_version: "encryptionkey@0"
I20260807 22:50:56.867293 18968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:56.915064 18968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:56.921556 18968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:56.921924 18968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:56.923322 18968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:56.925523 18968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:56.925634 18968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:56.925793 18968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:56.925916 18968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:56.967585 18968 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:34015
I20260807 22:50:56.967622 19103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:34015 every 8 connection(s)
I20260807 22:50:56.969539 18968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:56.970054 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 18968
I20260807 22:50:56.970278 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:50:56.971156 9995 external_mini_cluster.cc:1428] Setting key 8751070c4dfbb326b21d090b4be4ff73
Aug 07 22:50:56 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:57.002269 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:56.979644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37203 (local address 127.9.194.254:43971)
0807 22:50:56.980240 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:56.980249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:56.984604 (+ 4355us) server_negotiation.cc:407] Connection header received
0807 22:50:56.987696 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:56.987700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:56.987771 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:56.987912 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:56.989575 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.990426 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.992055 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.992467 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.992602 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:56.996176 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:56.996182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:56.996185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:56.996218 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:56.998885 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:56.999721 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:56.999725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:56.999728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:56.999849 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:57.000365 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:57.000374 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:57.000376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:57.000643 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:57.000808 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:57.001621 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 22:50:57.001846 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":320,"threads_started":1}
I20260807 22:50:57.003337 19106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:56.973570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:37203)
0807 22:50:56.984139 (+ 10569us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:56.984198 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:56.987274 (+ 3076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:56.988184 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:56.988193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:56.988794 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:56.989397 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:56.989416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.990593 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.990597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:56.991902 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:56.991921 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:56.992718 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:56.992728 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:56.992983 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:56.993703 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:56.993725 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:56.995953 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:56.999044 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:56.999054 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:56.999071 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:56.999588 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:57.000023 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:57.000029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:57.000040 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:57.000241 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:57.000775 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:57.000783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:57.001205 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 22:50:57.001501 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 22:50:57.002024 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10203,"thread_start_us":9985,"threads_started":1}
I20260807 22:50:57.004992 19104 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43971
I20260807 22:50:57.005311 19104 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:57.006048 19104 heartbeater.cc:511] Master 127.9.194.254:43971 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
I20260807 22:50:57.008488 18904 ts_manager.cc:194] Registered new tserver with Master: c243db2271dd4c9aa2004b08ecb7a561 (127.9.194.193:34015)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
I20260807 22:50:57.010442 18904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:37203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:50:57.048554 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:57.392568 19110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:57.392867 19110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:57.393031 19110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:57.400166 19110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:57.400270 19110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:57.400492 19110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:50:57.410043 19110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:57.411293 19110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:57.412595 19110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:57.425874 19116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:57.425925 19118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:57.425959 19115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:57.428946 19110 server_base.cc:1034] running on GCE node
I20260807 22:50:57.429587 19110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:57.430545 19110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:57.431871 19110 hybrid_clock.cc:648] HybridClock initialized: now 1786143057431816 us; error 55 us; skew 500 ppm
I20260807 22:50:57.432220 19110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:57 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143057, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:57.435523 19110 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:50:57.436957 19110 webserver.cc:460] Webserver started at http://127.9.194.194:34787/ using document root <none> and password file <none>
I20260807 22:50:57.437680 19110 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:57.437826 19110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:57.438138 19110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:57.441094 19110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "8d11ff21c3af4f12be3db1030897247e"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "86bb71b2e224226de0116a90b128a730"
server_key_iv: "22507e9ed072da3ffa3ba67c69f1a8a9"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.441916 19110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "8d11ff21c3af4f12be3db1030897247e"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "86bb71b2e224226de0116a90b128a730"
server_key_iv: "22507e9ed072da3ffa3ba67c69f1a8a9"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.447137 19110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:50:57.450531 19125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:57.451839 19110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:57.452018 19110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "8d11ff21c3af4f12be3db1030897247e"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "86bb71b2e224226de0116a90b128a730"
server_key_iv: "22507e9ed072da3ffa3ba67c69f1a8a9"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.452221 19110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:57.466033 19110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:57.469942 19110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:57.470232 19110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:57.471192 19110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:57.472577 19110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:57.472681 19110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:57.472829 19110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:57.472915 19110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:57.505909 19110 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:39949
I20260807 22:50:57.505960 19242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:39949 every 8 connection(s)
I20260807 22:50:57.507211 19110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:50:57.510254 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19110
I20260807 22:50:57.510489 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:50:57.511018 9995 external_mini_cluster.cc:1428] Setting key ac915b98c80e0847ca3b40ba9b028d1a
Aug 07 22:50:57 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143057, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:57.528507 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:57.509900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:54893 (local address 127.9.194.254:43971)
0807 22:50:57.510411 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:57.510417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:57.511513 (+ 1096us) server_negotiation.cc:407] Connection header received
0807 22:50:57.513415 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:57.513422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:57.513526 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:57.513644 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:57.515093 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:57.516285 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:57.517803 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:57.518538 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:57.518679 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:57.522070 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:57.522078 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:57.522086 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:57.522131 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:57.525245 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:57.526175 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:57.526178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:57.526180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:57.526306 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:57.526783 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:57.526786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:57.526788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:57.527042 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:57.527152 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:57.528072 (+ 920us) server_negotiation.cc:299] Negotiation successful
0807 22:50:57.528230 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260807 22:50:57.530085 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:57.510436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:54893)
0807 22:50:57.511331 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:57.511378 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:57.513151 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:57.513833 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:57.513841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:57.514357 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:57.514917 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:57.514933 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:57.516430 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:57.516434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:57.517660 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:57.517671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:57.518733 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:57.518737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:57.518919 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:57.519649 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:57.519670 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:57.521851 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:57.525379 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:57.525390 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:57.525408 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:57.526054 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:57.526417 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:57.526425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:57.526431 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:57.526678 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:57.527166 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:57.527177 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:57.527650 (+ 473us) client_negotiation.cc:769] Sending connection context
0807 22:50:57.528016 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 22:50:57.528584 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":357,"threads_started":1}
I20260807 22:50:57.531559 19243 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43971
I20260807 22:50:57.531872 19243 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:57.532598 19243 heartbeater.cc:511] Master 127.9.194.254:43971 requested a full tablet report, sending...
I20260807 22:50:57.534065 18904 ts_manager.cc:194] Registered new tserver with Master: 8d11ff21c3af4f12be3db1030897247e (127.9.194.194:39949)
I20260807 22:50:57.534996 18904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:54893
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:50:57.570410 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:57.893702 19249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:57.894027 19249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:57.894207 19249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:57.901216 19249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:57.901358 19249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:57.901544 19249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:50:57.910820 19249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:57.912056 19249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:57.913352 19249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:57.926538 19255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:57.926569 19256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:57.926786 19258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:57.927359 19249 server_base.cc:1034] running on GCE node
I20260807 22:50:57.927976 19249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:57.928853 19249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:57.930073 19249 hybrid_clock.cc:648] HybridClock initialized: now 1786143057930026 us; error 33 us; skew 500 ppm
I20260807 22:50:57.930429 19249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:57 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143057, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:57.933899 19249 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:50:57.935324 19249 webserver.cc:460] Webserver started at http://127.9.194.195:44865/ using document root <none> and password file <none>
I20260807 22:50:57.936074 19249 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:57.936220 19249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:57.936530 19249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:50:57.939383 19249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "8ffd1fbd95ce4a759ca94b2a28576076"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "3d83a65260e3a3bf79513f2445b13eb9"
server_key_iv: "4b2fb5b1dae39b9703d66b69a66ddc22"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.940352 19249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "8ffd1fbd95ce4a759ca94b2a28576076"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "3d83a65260e3a3bf79513f2445b13eb9"
server_key_iv: "4b2fb5b1dae39b9703d66b69a66ddc22"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.945569 19249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:50:57.948963 19265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:57.950434 19249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:50:57.950604 19249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "8ffd1fbd95ce4a759ca94b2a28576076"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "3d83a65260e3a3bf79513f2445b13eb9"
server_key_iv: "4b2fb5b1dae39b9703d66b69a66ddc22"
server_key_version: "encryptionkey@0"
I20260807 22:50:57.950799 19249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:57.965333 19249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:57.969483 19249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:57.969762 19249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:57.970696 19249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:57.972067 19249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:57.972169 19249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:57.972314 19249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:57.972394 19249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:58.005591 19249 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:40925
I20260807 22:50:58.005632 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:40925 every 8 connection(s)
I20260807 22:50:58.006915 19249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:50:58.014348 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19249
I20260807 22:50:58.014552 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:50:58.014551 19104 heartbeater.cc:503] Master 127.9.194.254:43971 was elected leader, sending a full tablet report...
I20260807 22:50:58.015069 9995 external_mini_cluster.cc:1428] Setting key 17a98c784ac98995537b150e6f9b1493
Aug 07 22:50:58 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143057, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:50:58.027642 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:58.009529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:47809 (local address 127.9.194.254:43971)
0807 22:50:58.009744 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:58.009750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:58.010949 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 22:50:58.012767 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:58.012772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:58.012858 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:58.012945 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:58.014607 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.015730 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.017937 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.018460 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.018559 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:58.022122 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:58.022126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:58.022130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:58.022160 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:58.024695 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.025398 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.025404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.025408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.025535 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.025955 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.025961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.025964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.026246 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:58.026417 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:58.027050 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 22:50:58.027270 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:50:58.028553 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:58.010106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:47809)
0807 22:50:58.010796 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:58.010840 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:58.012520 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:58.013147 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:58.013154 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:58.013650 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:58.014381 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:58.014406 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.015929 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.015934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:58.017794 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:58.017808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.018682 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.018687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:58.018896 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:58.019567 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:58.019590 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:50:58.021919 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:50:58.024859 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:58.024868 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:58.024884 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:58.025280 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:58.025652 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:50:58.025657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:50:58.025661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:50:58.025842 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:50:58.026380 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:50:58.026388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:50:58.026703 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 22:50:58.026954 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 22:50:58.027458 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":302,"threads_started":1}
I20260807 22:50:58.029886 19384 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43971
I20260807 22:50:58.030244 19384 heartbeater.cc:464] Registering TS with master...
I20260807 22:50:58.031028 19384 heartbeater.cc:511] Master 127.9.194.254:43971 requested a full tablet report, sending...
I20260807 22:50:58.032502 18904 ts_manager.cc:194] Registered new tserver with Master: 8ffd1fbd95ce4a759ca94b2a28576076 (127.9.194.195:40925)
I20260807 22:50:58.033279 18904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:47809
I20260807 22:50:58.042708 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
Entry for principal oryx/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab.
WARNING: no policy specified for oryx/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
Entry for principal oryx/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab.
WARNING: no policy specified for oryx/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
Entry for principal oryx/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab.
I20260807 22:50:58.200534 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18968
Aug 07 22:50:58 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:50:58.216858 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:58.201815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.9.194.193:34015)
0807 22:50:58.202763 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:58.202770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:58.202795 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:50:58.202856 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:58.202862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:58.203082 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:58.203212 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:58.204613 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.205638 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.206833 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.207390 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.207553 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:58.211605 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:58.211615 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:58.211630 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:58.211667 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:58.214296 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.214918 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.214924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.214926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.215047 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.215335 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.215338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.215339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.215776 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:58.216007 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:58.216244 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 22:50:58.216476 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":650,"threads_started":1}
I20260807 22:50:58.218151 19098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:58.310441 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18968
I20260807 22:50:58.329689 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19110
Aug 07 22:50:58 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:50:58.345599 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:58.330677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.9.194.194:39949)
0807 22:50:58.331262 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:58.331270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:58.331295 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:50:58.331363 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:58.331369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:58.331741 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:58.331897 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:58.333109 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.334132 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.335368 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.336003 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.336110 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:58.340242 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:58.340253 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:58.340267 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:58.340303 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:58.343135 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.343860 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.343868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.343870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.344002 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.344307 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.344310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.344312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.344628 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:58.344874 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:58.345116 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 22:50:58.345306 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":307,"threads_started":1}
I20260807 22:50:58.346907 19238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:58.430495 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19110
I20260807 22:50:58.449054 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19249
Aug 07 22:50:58 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:50:58.465754 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:58.450041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.9.194.195:40925)
0807 22:50:58.450566 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:58.450573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:58.450602 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:50:58.450662 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:58.450668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:58.450880 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:58.451054 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:58.452612 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.453587 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.454946 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:58.455625 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:58.455764 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:58.460216 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:50:58.460228 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:50:58.460241 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:50:58.460274 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:50:58.462905 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.463597 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.463605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.463608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.463726 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:50:58.464125 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:50:58.464129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:50:58.464131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:50:58.464468 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:50:58.464795 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:50:58.465115 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 22:50:58.465379 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260807 22:50:58.467096 19379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:58.549410 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19249
I20260807 22:50:58.567329 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 18871
I20260807 22:50:58.568573 18934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:50:58.671367 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 18871
I20260807 22:50:58.694468 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43971
--webserver_interface=127.9.194.254
--webserver_port=38091
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:50:59.024725 19401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:59.025094 19401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:59.025255 19401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:59.032552 19401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:50:59.032694 19401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:59.032841 19401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:59.032954 19401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:50:59.033036 19401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:50:59.042641 19401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=38091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:59.044011 19401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:59.045323 19401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:59.057518 19406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:59.057488 19407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:59.057709 19409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:59.058794 19401 server_base.cc:1034] running on GCE node
I20260807 22:50:59.059684 19401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:59.061120 19401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:59.062361 19401 hybrid_clock.cc:648] HybridClock initialized: now 1786143059062306 us; error 63 us; skew 500 ppm
I20260807 22:50:59.062726 19401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:59 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143059, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:59.066048 19401 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:50:59.067698 19401 webserver.cc:460] Webserver started at http://127.9.194.254:38091/ using document root <none> and password file <none>
I20260807 22:50:59.068367 19401 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:59.068483 19401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:59.073776 19401 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:50:59.076620 19416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:59.077996 19401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:50:59.078136 19401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "bb9b223e26c3388c97bd3d4dd0acc873"
server_key_iv: "259bd8ba73bf996fa2de4490a0c8f90d"
server_key_version: "encryptionkey@0"
I20260807 22:50:59.079058 19401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:59.097990 19401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:59.102800 19401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:59.103088 19401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:59.120911 19401 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43971
I20260807 22:50:59.120995 19470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43971 every 8 connection(s)
I20260807 22:50:59.122367 19401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:50:59.128497 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19401
I20260807 22:50:59.129281 19471 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:50:59.129307 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:34015
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:43971
--webserver_port=33859
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:50:59.132566 19471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Bootstrap starting.
I20260807 22:50:59.157109 19471 log.cc:824] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Log is configured to *not* fsync() on all Append() calls
I20260807 22:50:59.168569 19471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:50:59.169250 19471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Bootstrap complete.
I20260807 22:50:59.179013 19471 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:59.179936 19471 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35b3e72b6e49d6af7b72ef6f8bc1fe, State: Initialized, Role: FOLLOWER
I20260807 22:50:59.180611 19471 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:59.180855 19471 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:50:59.180976 19471 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:50:59.181170 19471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:50:59.185214 19471 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:59.185708 19471 leader_election.cc:302] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc35b3e72b6e49d6af7b72ef6f8bc1fe; no voters:
I20260807 22:50:59.186285 19471 leader_election.cc:288] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:50:59.186493 19475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:50:59.187691 19475 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [term 2 LEADER]: Becoming Leader. State: Replica: cc35b3e72b6e49d6af7b72ef6f8bc1fe, State: Running, Role: LEADER
I20260807 22:50:59.188370 19471 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:50:59.188434 19475 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } }
I20260807 22:50:59.195087 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } } }
I20260807 22:50:59.195472 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:59.196159 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc35b3e72b6e49d6af7b72ef6f8bc1fe. Latest consensus state: current_term: 2 leader_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc35b3e72b6e49d6af7b72ef6f8bc1fe" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43971 } } }
I20260807 22:50:59.196511 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe [sys.catalog]: This master's current role is: LEADER
I20260807 22:50:59.203171 19492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:50:59.208575 19492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:50:59.210466 19492 catalog_manager.cc:1259] Loaded cluster ID: 7455dfc59ff24dbaac1f4e1ec9097eab
I20260807 22:50:59.210570 19492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:50:59.214752 19492 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:50:59.216187 19492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cc35b3e72b6e49d6af7b72ef6f8bc1fe: Loaded TSK: 0
I20260807 22:50:59.217638 19492 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:50:59.486202 19473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:59.486541 19473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:59.486714 19473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:50:59.494807 19473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:50:59.494974 19473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:50:59.495159 19473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:50:59.505427 19473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.193.keytab
--principal=oryx/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:34015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=33859
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:50:59.506896 19473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:50:59.508306 19473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:50:59.522985 19499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:59.523751 19502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:50:59.523916 19500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:50:59.524978 19473 server_base.cc:1034] running on GCE node
I20260807 22:50:59.525605 19473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:50:59.526594 19473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:50:59.527858 19473 hybrid_clock.cc:648] HybridClock initialized: now 1786143059527788 us; error 41 us; skew 500 ppm
I20260807 22:50:59.528188 19473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:50:59 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143059, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:50:59.536782 19473 init.cc:377] Logged in from keytab as oryx/127.9.194.193@KRBTEST.COM (short username oryx)
I20260807 22:50:59.538432 19473 webserver.cc:460] Webserver started at http://127.9.194.193:33859/ using document root <none> and password file <none>
I20260807 22:50:59.539142 19473 fs_manager.cc:359] Metadata directory not provided
I20260807 22:50:59.539288 19473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:50:59.544816 19473 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 22:50:59.547681 19509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:59.549088 19473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:50:59.549244 19473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "c243db2271dd4c9aa2004b08ecb7a561"
format_stamp: "Formatted at 2026-08-07 22:50:56 on dist-test-slave-5k54"
server_key: "ad7b2d2667d1990c9837232161ced559"
server_key_iv: "4c88d4285143d9ef65d14c4446eed5d6"
server_key_version: "encryptionkey@0"
I20260807 22:50:59.550196 19473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:50:59.566227 19473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:50:59.570782 19473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:50:59.571096 19473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:50:59.572257 19473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:50:59.573596 19473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:50:59.573685 19473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:59.573849 19473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:50:59.573951 19473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:50:59.608276 19473 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:34015
I20260807 22:50:59.608363 19630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:34015 every 8 connection(s)
I20260807 22:50:59.609757 19473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:50:59.611488 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19473
I20260807 22:50:59.612174 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:39949
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:43971
--webserver_port=34787
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:50:59 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:50:59 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:50:59.993427 19635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:50:59.993738 19635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:50:59.993899 19635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:00.001061 19635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:00.001204 19635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:00.001376 19635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:51:00.011386 19635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.194.keytab
--principal=oryx/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:39949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=34787
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:00.012924 19635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:00.014318 19635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:00.030357 19642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:00.030946 19644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:00.031030 19641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:00.037453 19635 server_base.cc:1034] running on GCE node
I20260807 22:51:00.038623 19635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:00.040360 19635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:00.041738 19635 hybrid_clock.cc:648] HybridClock initialized: now 1786143060041646 us; error 57 us; skew 500 ppm
I20260807 22:51:00.042272 19635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:00 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143060, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:00.047658 19635 init.cc:377] Logged in from keytab as oryx/127.9.194.194@KRBTEST.COM (short username oryx)
I20260807 22:51:00.049518 19635 webserver.cc:460] Webserver started at http://127.9.194.194:34787/ using document root <none> and password file <none>
I20260807 22:51:00.050238 19635 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:00.050406 19635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:00.057143 19635 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:51:00.060123 19652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.061630 19635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:51:00.061806 19635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "8d11ff21c3af4f12be3db1030897247e"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "86bb71b2e224226de0116a90b128a730"
server_key_iv: "22507e9ed072da3ffa3ba67c69f1a8a9"
server_key_version: "encryptionkey@0"
I20260807 22:51:00.062773 19635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:00.097347 19635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:00.102029 19635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:00.102317 19635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:00.103473 19635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:00.104951 19635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:00.105070 19635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.105206 19635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:00.105281 19635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.154013 19635 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:39949
I20260807 22:51:00.154127 19766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:39949 every 8 connection(s)
I20260807 22:51:00.155608 19635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:51:00.162151 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19635
I20260807 22:51:00.162907 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:40925
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:43971
--webserver_port=44865
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:51:00 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:00 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:51:00.815958 19772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:00.816287 19772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:00.816490 19772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:00.824465 19772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:00.824595 19772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:00.824808 19772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:51:00.835081 19772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/krb5kdc/oryx_127.9.194.195.keytab
--principal=oryx/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:40925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=44865
--tserver_master_addrs=127.9.194.254:43971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:00.836596 19772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:00.838047 19772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:00.852680 19779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:00.852680 19780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:00.853574 19782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:00.854532 19772 server_base.cc:1034] running on GCE node
I20260807 22:51:00.855146 19772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:00.856367 19772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:00.857635 19772 hybrid_clock.cc:648] HybridClock initialized: now 1786143060857575 us; error 54 us; skew 500 ppm
I20260807 22:51:00.857959 19772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:00 dist-test-slave-5k54 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143060, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:00.861459 19772 init.cc:377] Logged in from keytab as oryx/127.9.194.195@KRBTEST.COM (short username oryx)
I20260807 22:51:00.863977 19772 webserver.cc:460] Webserver started at http://127.9.194.195:44865/ using document root <none> and password file <none>
I20260807 22:51:00.864715 19772 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:00.864877 19772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:00.875113 19772 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.003s
I20260807 22:51:00.881335 19790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.882930 19772 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.004s
I20260807 22:51:00.883113 19772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "8ffd1fbd95ce4a759ca94b2a28576076"
format_stamp: "Formatted at 2026-08-07 22:50:57 on dist-test-slave-5k54"
server_key: "3d83a65260e3a3bf79513f2445b13eb9"
server_key_iv: "4b2fb5b1dae39b9703d66b69a66ddc22"
server_key_version: "encryptionkey@0"
I20260807 22:51:00.884210 19772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:00.901723 19772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:00.906075 19772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:00.906344 19772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:00.907382 19772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:00.908799 19772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:00.908937 19772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.909075 19772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:00.909162 19772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:00.967993 19772 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:40925
I20260807 22:51:00.968142 19911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:40925 every 8 connection(s)
I20260807 22:51:00.969888 19772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:51:00.972793 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19772
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:01.012904 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:00.974193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.9.194.254:43971)
0807 22:51:00.990723 (+ 16530us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:00.990733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:00.990766 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:51:00.990859 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:00.990867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:00.990980 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:00.991152 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:00.993548 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.995382 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.997438 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.998359 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.998549 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.001989 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:01.002001 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:01.002031 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:01.002108 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:01.006823 (+ 4715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:01.008045 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:01.008054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:01.008064 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:01.008274 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:01.008823 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:01.008829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:01.008832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:01.009474 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:01.009753 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:01.010686 (+ 933us) server_negotiation.cc:299] Negotiation successful
0807 22:51:01.011350 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16337,"thread_start_us":16157,"threads_started":1}
W20260807 22:51:01.021514 19634 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.396s user 0.001s sys 0.012s
W20260807 22:51:01.023344 19633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:50:59.613232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:46707 (local address 127.9.194.254:43971)
0807 22:50:59.616172 (+ 2940us) server_negotiation.cc:206] Beginning negotiation
0807 22:50:59.616183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:50:59.616224 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:50:59.617670 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:50:59.617683 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:50:59.618018 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:50:59.618305 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:50:59.620366 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:59.621616 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:59.623399 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:50:59.624111 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:50:59.624245 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.022966 (+1398721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:46707: BlockingRecv error: recv got EOF from 127.9.194.193:46707 (error 108)
Metrics: {"server-negotiator.queue_time_us":2705,"thread_start_us":2503,"threads_started":1}
I20260807 22:51:01.024232 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:50:59.613237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:46707)
0807 22:50:59.614167 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0807 22:50:59.614220 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:50:59.617117 (+ 2897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:50:59.618531 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:50:59.618545 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:50:59.619354 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:50:59.620174 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:59.620202 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:59.621800 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:59.621807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:59.623250 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:50:59.623268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:50:59.624356 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:50:59.624366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:50:59.624693 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:50:59.625486 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:50:59.625527 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.022695 (+1397168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":386,"threads_started":1}
W20260807 22:51:01.025283 19631 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:43971 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.194.254:43971: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:01.036157 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:01.026105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:57651)
0807 22:51:01.026723 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:01.026744 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:01.026872 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:01.027307 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:01.027314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:01.027832 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:01.028059 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.028070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.029066 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.029071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.030327 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.030340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.031472 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.031480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.031611 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.032244 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:01.032263 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.035855 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476}
W20260807 22:51:01.036288 19633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:01.026364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:57651 (local address 127.9.194.254:43971)
0807 22:51:01.026552 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:01.026560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:01.026838 (+ 278us) server_negotiation.cc:407] Connection header received
0807 22:51:01.026999 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:01.027006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:01.027121 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:01.027266 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:01.028206 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.028918 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.030467 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.031029 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.031161 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.036044 (+ 4883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:57651: BlockingRecv error: recv got EOF from 127.9.194.193:57651 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:51:01.046933 19633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:01.037216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:44645 (local address 127.9.194.254:43971)
0807 22:51:01.037381 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:01.037386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:01.037486 (+ 100us) server_negotiation.cc:407] Connection header received
0807 22:51:01.037663 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:01.037667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:01.037747 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:01.037846 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:01.038560 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.039242 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.040506 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.040940 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.041073 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.046682 (+ 5609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:44645: BlockingRecv error: recv got EOF from 127.9.194.193:44645 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:51:01.047379 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:01.037049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:44645)
0807 22:51:01.037392 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:01.037411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:01.037539 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:01.037878 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:01.037883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:01.038280 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:01.038427 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.038435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.039382 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.039387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.040366 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.040377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.041096 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.041101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.041280 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.041868 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:01.041890 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.047196 (+ 5306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245}
W20260807 22:51:01.047807 19631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:51:01.369743 19771 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.196s user 0.001s sys 0.014s
W20260807 22:51:01.371733 19770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:00.158706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:42275 (local address 127.9.194.254:43971)
0807 22:51:00.159190 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:00.159194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:00.161095 (+ 1901us) server_negotiation.cc:407] Connection header received
0807 22:51:00.163838 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:00.163848 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:00.164002 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:00.164181 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:00.166390 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.168266 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.170853 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.171881 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.172068 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.371288 (+1199220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:42275: BlockingRecv error: recv got EOF from 127.9.194.194:42275 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":258,"threads_started":1}
I20260807 22:51:01.372843 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:00.159884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:42275)
0807 22:51:00.160921 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:00.160960 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:00.163477 (+ 2517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:00.164466 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:00.164490 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:00.165317 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:00.166151 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:00.166180 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.168495 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.168503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:00.170678 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:00.170696 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.172151 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.172161 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:00.172500 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:00.173426 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:00.173455 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.371080 (+1197625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":390,"threads_started":1}
W20260807 22:51:01.373844 19767 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:43971 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.194.254:43971: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:51:01.384465 19770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:01.374809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:43151 (local address 127.9.194.254:43971)
0807 22:51:01.374997 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:01.375002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:01.375014 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:51:01.375053 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:01.375056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:01.375110 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:01.375208 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:01.375983 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.376706 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.377954 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.378344 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.378475 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.384233 (+ 5758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:43151: BlockingRecv error: recv got EOF from 127.9.194.194:43151 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:51:01.384486 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:01.374510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:43151)
0807 22:51:01.374691 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:01.374710 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:01.374807 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:01.375239 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:01.375244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:01.375683 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:01.375849 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.375857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.376850 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.376856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.377821 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.377832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.378510 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.378517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.378691 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.379287 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:01.379304 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.384053 (+ 4749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:01 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:01.397120 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:01.385377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:39653)
0807 22:51:01.385574 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:01.385592 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:01.385689 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:01.386226 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:01.386229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:01.386515 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:01.386658 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.386665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.388736 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.388741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.389808 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.389819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.391194 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.391198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.391377 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.392031 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:01.392111 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:01.396697 (+ 4586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260807 22:51:01.397481 19767 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:51:01.398116 19770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:01.385587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:39653 (local address 127.9.194.254:43971)
0807 22:51:01.385821 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:01.385830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:01.385857 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:51:01.385926 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:01.385934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:01.386068 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:01.386210 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:01.386804 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.388536 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.390013 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.391030 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.391205 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.397720 (+ 6515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:39653: BlockingRecv error: recv got EOF from 127.9.194.194:39653 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:02.063012 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:02.048431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:58357)
0807 22:51:02.050587 (+ 2156us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:02.050618 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:02.050808 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:02.051194 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:02.051202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:02.052060 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:02.052355 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.052372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.053431 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.053439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.055914 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.055939 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.056989 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.057000 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.057277 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.058071 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:02.058108 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:02.062766 (+ 4658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1961,"thread_start_us":1755,"threads_started":1}
W20260807 22:51:02.063722 19922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:02.048523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58357 (local address 127.9.194.254:43971)
0807 22:51:02.050815 (+ 2292us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:02.050819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:02.050836 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:51:02.050954 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:02.050958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:02.051019 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:02.051106 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:02.052531 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.053270 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.056097 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.056716 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.056863 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.063188 (+ 6325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:58357: BlockingRecv error: recv got EOF from 127.9.194.193:58357 (error 108)
Metrics: {"server-negotiator.queue_time_us":2201,"thread_start_us":2070,"threads_started":1}
W20260807 22:51:02.181867 19917 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.174s user 0.004s sys 0.014s
W20260807 22:51:02.183143 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:00.990531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:36665 (local address 127.9.194.254:43971)
0807 22:51:00.992180 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:00.992189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:00.992214 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:51:00.994098 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:00.994107 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:00.994189 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:00.994293 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:00.996172 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.999323 (+ 3151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.004650 (+ 5327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.005738 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.005902 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.182913 (+1177011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:36665: BlockingRecv error: recv got EOF from 127.9.194.195:36665 (error 108)
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":1459,"thread_start_us":1214,"threads_started":1}
I20260807 22:51:02.183776 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:00.975776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:36665)
0807 22:51:00.989332 (+ 13556us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:00.989392 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:00.993825 (+ 4433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:00.994630 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:00.994638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:00.995169 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:00.995946 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:00.995971 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:00.999633 (+ 3662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:00.999643 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.004354 (+ 4711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:01.004378 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:01.006423 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:01.006433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:01.006903 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:01.007818 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:01.007872 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:02.182683 (+1174811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13221,"thread_start_us":12999,"threads_started":1}
W20260807 22:51:02.184665 19912 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:43971 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.194.254:43971: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:02.194267 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:02.185335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:57945)
0807 22:51:02.185539 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:02.185562 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:02.185669 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:02.185942 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:02.185945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:02.186281 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:02.186456 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.186463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.187502 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.187506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.188452 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.188463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.189322 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.189332 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.189466 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.190142 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:02.190170 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:02.193956 (+ 3786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 22:51:02.194410 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:02.185449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:57945 (local address 127.9.194.254:43971)
0807 22:51:02.185677 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:02.185682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:02.185699 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:51:02.185743 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:02.185746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:02.185803 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:02.185881 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:02.186572 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.187338 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.188593 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.189142 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.189236 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.194163 (+ 4927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:57945: BlockingRecv error: recv got EOF from 127.9.194.195:57945 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:02.203272 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:02.195019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:56615)
0807 22:51:02.195174 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:02.195192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:02.195277 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:02.195597 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:02.195600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:02.195864 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:02.196045 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.196052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.196963 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.196966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.197900 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.197911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.198545 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.198549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.198664 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.199191 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:02.199207 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:02.202997 (+ 3790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 22:51:02.203477 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:02.195121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:56615 (local address 127.9.194.254:43971)
0807 22:51:02.195259 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:02.195263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:02.195278 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:51:02.195361 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:02.195364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:02.195418 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:02.195552 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:02.196158 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.196827 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.198029 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.198420 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.198510 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.203192 (+ 4682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:56615: BlockingRecv error: recv got EOF from 127.9.194.195:56615 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 22:51:02.203630 19912 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:02 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:02.407392 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:02.398250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:56893)
0807 22:51:02.398824 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:02.398840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:02.398951 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:02.399269 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:02.399272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:02.399784 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:02.399936 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.399944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.400916 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.400920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.402014 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:02.402026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.402948 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.402952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:02.403135 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.403813 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:02.403831 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:02.407162 (+ 3331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":349,"threads_started":1}
W20260807 22:51:02.407676 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:02.398365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:56893 (local address 127.9.194.254:43971)
0807 22:51:02.398566 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:02.398570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:02.398917 (+ 347us) server_negotiation.cc:407] Connection header received
0807 22:51:02.399067 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:02.399071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:02.399146 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:02.399221 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:02.400080 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.400787 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.402152 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:02.402770 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:02.402900 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:02.407361 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:56893: BlockingRecv error: recv got EOF from 127.9.194.194:56893 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:03.073446 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:03.063983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:51235)
0807 22:51:03.064541 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:03.064558 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:03.064663 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:03.064953 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:03.064956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:03.065340 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:03.065628 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.065639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.066728 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.066732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.067823 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.067835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.068508 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.068512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.068632 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.069282 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:03.069299 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:03.073072 (+ 3773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":271,"threads_started":1}
W20260807 22:51:03.073536 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:03.064074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:51235 (local address 127.9.194.254:43971)
0807 22:51:03.064604 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:03.064608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:03.064624 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:51:03.064747 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:03.064751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:03.064820 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:03.064905 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:03.065769 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.066619 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.067938 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.068368 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.068460 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.073271 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:51235: BlockingRecv error: recv got EOF from 127.9.194.193:51235 (error 108)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":281,"threads_started":1}
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:03.213384 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:03.204233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:35113)
0807 22:51:03.204836 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:03.204853 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:03.204968 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:03.205253 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:03.205256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:03.205719 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:03.205867 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.205876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.206866 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.206869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.207872 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.207884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.208586 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.208590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.208706 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.209328 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:03.209346 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:03.213145 (+ 3799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":363,"threads_started":1}
W20260807 22:51:03.213711 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:03.204277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:35113 (local address 127.9.194.254:43971)
0807 22:51:03.204461 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:03.204466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:03.204927 (+ 461us) server_negotiation.cc:407] Connection header received
0807 22:51:03.205065 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:03.205068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:03.205129 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:03.205206 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:03.206000 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.206753 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.208002 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.208436 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.208540 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.213397 (+ 4857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:35113: BlockingRecv error: recv got EOF from 127.9.194.195:35113 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:03 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:03.417253 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:03.408540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:52349)
0807 22:51:03.409044 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:03.409061 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:03.409184 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:03.409464 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:03.409467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:03.409868 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:03.410062 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.410071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.411048 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.411051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.412057 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:03.412069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.412693 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.412697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:03.412812 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.413541 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:03.413565 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:03.417020 (+ 3455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":294,"threads_started":1}
W20260807 22:51:03.417459 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:03.408655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:52349 (local address 127.9.194.254:43971)
0807 22:51:03.408925 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:03.408931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:03.409138 (+ 207us) server_negotiation.cc:407] Connection header received
0807 22:51:03.409277 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:03.409281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:03.409342 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:03.409422 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:03.410186 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.410928 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.412174 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:03.412565 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:03.412658 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:03.417207 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:52349: BlockingRecv error: recv got EOF from 127.9.194.194:52349 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:04.083458 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:04.074364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:60767)
0807 22:51:04.074912 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:04.074928 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:04.075048 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:04.075325 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:04.075328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:04.075741 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:04.075918 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.075927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.076965 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.076969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.077869 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.077880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.078572 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.078577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.078749 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.079475 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:04.079515 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:04.083122 (+ 3607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":302,"threads_started":1}
W20260807 22:51:04.083699 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:04.074546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:60767 (local address 127.9.194.254:43971)
0807 22:51:04.075041 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:04.075045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:04.075062 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:51:04.075144 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:04.075147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:04.075204 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:04.075281 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:04.076056 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.076822 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.077988 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.078405 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.078539 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.083373 (+ 4834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:60767: BlockingRecv error: recv got EOF from 127.9.194.193:60767 (error 108)
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":263,"threads_started":1}
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:04.223809 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:04.214354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:34295)
0807 22:51:04.214965 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:04.214982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:04.215114 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:04.215386 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:04.215389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:04.215813 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:04.216022 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.216031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.216994 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.216998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.217973 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.217993 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.218698 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.218704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.218973 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.219664 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:04.219682 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:04.223560 (+ 3878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":366,"threads_started":1}
W20260807 22:51:04.223892 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:04.214400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:34295 (local address 127.9.194.254:43971)
0807 22:51:04.214574 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:04.214578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:04.215055 (+ 477us) server_negotiation.cc:407] Connection header received
0807 22:51:04.215200 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:04.215204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:04.215262 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:04.215340 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:04.216145 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.216877 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.218123 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.218522 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.218615 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.223725 (+ 5110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:34295: BlockingRecv error: recv got EOF from 127.9.194.195:34295 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:04 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:04.427471 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:04.418414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:60961)
0807 22:51:04.418904 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:04.418938 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:04.419056 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:04.419353 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:04.419356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:04.419743 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:04.419933 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.419942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.420923 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.420927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.421852 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:04.421864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.422537 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.422541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:04.422663 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.423319 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:04.423338 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:04.427157 (+ 3819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
W20260807 22:51:04.427773 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:04.418496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:60961 (local address 127.9.194.254:43971)
0807 22:51:04.418743 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:04.418752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:04.419041 (+ 289us) server_negotiation.cc:407] Connection header received
0807 22:51:04.419153 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:04.419157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:04.419233 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:04.419308 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:04.420061 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.420785 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.421986 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:04.422418 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:04.422517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:04.427497 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:60961: BlockingRecv error: recv got EOF from 127.9.194.194:60961 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:05.102221 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:05.084401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:48383)
0807 22:51:05.084968 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:05.084985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:05.085089 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:05.085408 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:05.085411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:05.085790 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:05.085956 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.085965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.087100 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.087106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.088355 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.088367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.089237 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.089243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.089431 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.098178 (+ 8747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:05.098197 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:05.101955 (+ 3758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":304,"threads_started":1}
W20260807 22:51:05.102471 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:05.084509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:48383 (local address 127.9.194.254:43971)
0807 22:51:05.085066 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:05.085070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:05.085087 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:51:05.085200 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:05.085204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:05.085265 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:05.085356 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:05.086087 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.086959 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.088489 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.089093 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.089215 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.102194 (+ 12979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:48383: BlockingRecv error: recv got EOF from 127.9.194.193:48383 (error 108)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:05.233686 19934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:05.224833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:33669)
0807 22:51:05.225396 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:05.225413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:05.225529 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:05.225820 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:05.225823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:05.226182 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:05.226356 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.226364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.227403 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.227407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.228411 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.228424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.229047 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.229071 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.229200 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.229908 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:05.229927 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:05.233447 (+ 3520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":314,"threads_started":1}
W20260807 22:51:05.233879 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:05.224905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:33669 (local address 127.9.194.254:43971)
0807 22:51:05.225137 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:05.225142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:05.225485 (+ 343us) server_negotiation.cc:407] Connection header received
0807 22:51:05.225625 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:05.225629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:05.225690 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:05.225770 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:05.226487 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.227233 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.228531 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.228911 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.229005 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.233644 (+ 4639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:33669: BlockingRecv error: recv got EOF from 127.9.194.195:33669 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:05 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:05.437700 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:05.428485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:38645)
0807 22:51:05.429073 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:05.429090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:05.429202 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:05.429517 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:05.429520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:05.429899 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:05.430063 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.430071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.431052 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.431055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.432281 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:05.432298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.433263 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.433267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:05.433398 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.434015 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:05.434032 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:05.437463 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":335,"threads_started":1}
W20260807 22:51:05.437853 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:05.428609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:38645 (local address 127.9.194.254:43971)
0807 22:51:05.428795 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:05.428800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:05.429170 (+ 370us) server_negotiation.cc:407] Connection header received
0807 22:51:05.429300 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:05.429303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:05.429379 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:05.429459 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:05.430183 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.430943 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.432439 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:05.433123 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:05.433225 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:05.437636 (+ 4411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:38645: BlockingRecv error: recv got EOF from 127.9.194.194:38645 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:06.112507 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:06.103282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:60465)
0807 22:51:06.103883 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:06.103900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:06.104005 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:06.104325 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:06.104329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:06.104779 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:06.104994 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.105004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.106051 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.106055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.107038 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.107050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.107707 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.107710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.107833 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.108419 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:06.108436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:06.112177 (+ 3741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":350,"threads_started":1}
W20260807 22:51:06.112645 19938 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:06.103385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:60465 (local address 127.9.194.254:43971)
0807 22:51:06.103941 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:06.103945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:06.103961 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:51:06.104092 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:06.104096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:06.104165 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:06.104250 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:06.105152 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.105908 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.107163 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.107590 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.107677 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.112385 (+ 4708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:60465: BlockingRecv error: recv got EOF from 127.9.194.193:60465 (error 108)
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":297,"threads_started":1}
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:06.243753 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:06.234702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:54203)
0807 22:51:06.235293 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:06.235311 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:06.235419 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:06.235738 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:06.235741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:06.236155 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:06.236324 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.236333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.237324 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.237327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.238518 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.238531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.239184 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.239188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.239307 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.239981 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:06.240010 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:06.243343 (+ 3333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":348,"threads_started":1}
W20260807 22:51:06.243831 19938 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:06.234822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:54203 (local address 127.9.194.254:43971)
0807 22:51:06.235012 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:06.235017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:06.235387 (+ 370us) server_negotiation.cc:407] Connection header received
0807 22:51:06.235539 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:06.235543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:06.235604 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:06.235687 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:06.236451 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.237213 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.238647 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.239061 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.239153 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.243582 (+ 4429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:54203: BlockingRecv error: recv got EOF from 127.9.194.195:54203 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:06 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:06.455597 19940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:06.438703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:55055)
0807 22:51:06.439288 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:06.439304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:06.439493 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:06.439898 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:06.439903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:06.440337 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:06.440481 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.440489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.441507 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.441512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.442476 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:06.442488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.443402 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.443407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:06.443611 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.450562 (+ 6951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:06.450583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:06.455249 (+ 4666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":343,"threads_started":1}
W20260807 22:51:06.455716 19938 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:06.438951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:55055 (local address 127.9.194.254:43971)
0807 22:51:06.439196 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:06.439203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:06.439387 (+ 184us) server_negotiation.cc:407] Connection header received
0807 22:51:06.439631 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:06.439636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:06.439722 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:06.439823 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:06.440627 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.441379 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.442612 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:06.443261 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:06.443379 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:06.455340 (+ 11961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:55055: BlockingRecv error: recv got EOF from 127.9.194.194:55055 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:07.123068 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:07.113547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:37779)
0807 22:51:07.114031 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:07.114048 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:07.114154 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:07.114692 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:07.114695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:07.115088 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:07.115234 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.115242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.116275 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.116278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.117592 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.117607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.118474 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.118480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.118621 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.119208 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:07.119226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:07.122846 (+ 3620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":266,"threads_started":1}
W20260807 22:51:07.123335 19943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:07.113794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37779 (local address 127.9.194.254:43971)
0807 22:51:07.114427 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:07.114431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:07.114447 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:51:07.114493 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:07.114496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:07.114558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:07.114646 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:07.115366 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.116155 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.117730 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.118302 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.118453 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.123075 (+ 4622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:37779: BlockingRecv error: recv got EOF from 127.9.194.193:37779 (error 108)
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":395,"threads_started":1}
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:07.254282 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:07.244854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:43767)
0807 22:51:07.245455 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:07.245472 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:07.245632 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:07.245968 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:07.245974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:07.246435 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:07.246587 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.246596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.247750 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.247756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.248693 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.248704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.249396 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.249401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.249580 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.250168 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:07.250186 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:07.254033 (+ 3847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":306,"threads_started":1}
W20260807 22:51:07.254423 19943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:07.245019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:43767 (local address 127.9.194.254:43971)
0807 22:51:07.245266 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:07.245273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:07.245556 (+ 283us) server_negotiation.cc:407] Connection header received
0807 22:51:07.245770 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:07.245773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:07.245837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:07.245932 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:07.246725 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.247624 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.248844 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.249249 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.249384 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.254223 (+ 4839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:43767: BlockingRecv error: recv got EOF from 127.9.194.195:43767 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:07 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:07.465610 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:07.456632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:40055)
0807 22:51:07.457092 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:07.457126 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:07.457243 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:07.457538 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:07.457541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:07.457984 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:07.458159 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.458167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.459138 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.459142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.460111 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:07.460123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.460980 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.460988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:07.461141 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.461870 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:07.461892 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:07.465387 (+ 3495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":269,"threads_started":1}
W20260807 22:51:07.465919 19943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:07.456693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:40055 (local address 127.9.194.254:43971)
0807 22:51:07.456939 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:07.456945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:07.457205 (+ 260us) server_negotiation.cc:407] Connection header received
0807 22:51:07.457336 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:07.457340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:07.457416 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:07.457495 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:07.458275 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.459009 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.460229 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:07.460799 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:07.460901 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:07.465587 (+ 4686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:40055: BlockingRecv error: recv got EOF from 127.9.194.194:40055 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:08.145439 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:08.124348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:58709)
0807 22:51:08.125024 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:08.125046 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:08.125169 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:08.136817 (+ 11648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:08.136824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:08.137408 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:08.137716 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.137733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.138789 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.138794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.139884 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.139896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.140698 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.140703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.140875 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.141518 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:08.141536 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:08.145087 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":271,"threads_started":1}
W20260807 22:51:08.145574 19947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:08.125047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58709 (local address 127.9.194.254:43971)
0807 22:51:08.136443 (+ 11396us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:08.136451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:08.136473 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:51:08.136548 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:08.136555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:08.136643 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:08.136770 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:08.137872 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.138642 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.140026 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.140531 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.140685 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.145280 (+ 4595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:58709: BlockingRecv error: recv got EOF from 127.9.194.193:58709 (error 108)
Metrics: {"server-negotiator.queue_time_us":11260,"thread_start_us":11103,"threads_started":1}
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:08.265910 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:08.255266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:37347)
0807 22:51:08.255867 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:08.255884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:08.255995 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:08.256342 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:08.256345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:08.256750 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:08.256927 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.256935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.258561 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.258565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.259594 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.259626 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.261020 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.261027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.261219 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.261874 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:08.261891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:08.265555 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":366,"threads_started":1}
W20260807 22:51:08.266137 19947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:08.255385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:37347 (local address 127.9.194.254:43971)
0807 22:51:08.255617 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:08.255626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:08.255965 (+ 339us) server_negotiation.cc:407] Connection header received
0807 22:51:08.256093 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:08.256102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:08.256199 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:08.256314 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:08.257067 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.258398 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.259786 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.260688 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.260844 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.265844 (+ 5000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:37347: BlockingRecv error: recv got EOF from 127.9.194.195:37347 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:08 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:08.475917 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:08.466528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:33707)
0807 22:51:08.467175 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:08.467192 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:08.467305 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:08.467797 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:08.467801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:08.468296 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:08.468494 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.468504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.469509 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.469512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.470451 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:08.470462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.471196 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.471201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:08.471321 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.471933 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:08.471953 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:08.475578 (+ 3625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":332,"threads_started":1}
W20260807 22:51:08.476022 19947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:08.466654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:33707 (local address 127.9.194.254:43971)
0807 22:51:08.466833 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:08.466838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:08.467265 (+ 427us) server_negotiation.cc:407] Connection header received
0807 22:51:08.467573 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:08.467581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:08.467663 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:08.467749 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:08.468624 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.469393 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.470596 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:08.471054 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:08.471152 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:08.475763 (+ 4611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:33707: BlockingRecv error: recv got EOF from 127.9.194.194:33707 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:09.157644 19950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:09.148375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:47941)
0807 22:51:09.148881 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:09.148898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:09.149000 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:09.149650 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:09.149654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:09.150196 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:09.150443 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.150458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.151553 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.151556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.152486 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.152499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.153188 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.153192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.153303 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.153974 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:09.154001 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:09.157396 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":266,"threads_started":1}
W20260807 22:51:09.157910 19952 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:09.148466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:47941 (local address 127.9.194.254:43971)
0807 22:51:09.149301 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:09.149307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:09.149331 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:51:09.149378 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:09.149381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:09.149480 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:09.149589 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:09.150614 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.151393 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.152607 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.153041 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.153130 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.157627 (+ 4497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:47941: BlockingRecv error: recv got EOF from 127.9.194.193:47941 (error 108)
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":538,"threads_started":1}
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:09.275962 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:09.267041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:58531)
0807 22:51:09.267717 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:09.267740 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:09.267880 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:09.268168 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:09.268171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:09.268552 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:09.268719 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.268727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.269709 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.269713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.270680 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.270691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.271386 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.271391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.271574 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.272208 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:09.272226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:09.275734 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":346,"threads_started":1}
W20260807 22:51:09.276151 19952 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:09.267229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:58531 (local address 127.9.194.254:43971)
0807 22:51:09.267563 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:09.267573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:09.267815 (+ 242us) server_negotiation.cc:407] Connection header received
0807 22:51:09.267990 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:09.267993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:09.268055 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:09.268124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:09.268848 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.269597 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.270815 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.271256 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.271351 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.275931 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:58531: BlockingRecv error: recv got EOF from 127.9.194.195:58531 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:09 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:09.486465 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:09.476962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:49895)
0807 22:51:09.477480 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:09.477510 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:09.477677 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:09.478011 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:09.478015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:09.478572 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:09.478772 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.478785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.480065 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.480071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.481002 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:09.481012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.481781 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.481788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:09.481946 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.482632 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:09.482650 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:09.486123 (+ 3473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":278,"threads_started":1}
W20260807 22:51:09.486567 19952 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:09.477037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:49895 (local address 127.9.194.254:43971)
0807 22:51:09.477269 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:09.477277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:09.477595 (+ 318us) server_negotiation.cc:407] Connection header received
0807 22:51:09.477803 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:09.477808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:09.477879 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:09.477977 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:09.478914 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.479909 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.481146 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:09.481590 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:09.481713 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:09.486324 (+ 4611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:49895: BlockingRecv error: recv got EOF from 127.9.194.194:49895 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:10.173259 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:10.158667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:40819)
0807 22:51:10.159808 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:10.159824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:10.159975 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:10.160345 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:10.160349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:10.160803 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:10.160957 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.160966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.162027 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.162033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.163000 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.163011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.163883 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.163889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.164088 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.169027 (+ 4939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:10.169045 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:10.172938 (+ 3893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1002,"thread_start_us":695,"threads_started":1}
W20260807 22:51:10.173383 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:10.158697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:40819 (local address 127.9.194.254:43971)
0807 22:51:10.159460 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:10.159470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:10.159904 (+ 434us) server_negotiation.cc:407] Connection header received
0807 22:51:10.160112 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:10.160131 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:10.160200 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:10.160310 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:10.161109 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.161881 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.163148 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.163733 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.163865 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.173125 (+ 9260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:40819: BlockingRecv error: recv got EOF from 127.9.194.193:40819 (error 108)
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":352,"threads_started":1}
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:10.286350 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:10.276911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:47905)
0807 22:51:10.277530 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:10.277551 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:10.277742 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:10.278143 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:10.278149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:10.278715 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:10.278915 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.278924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.279971 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.279975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.281081 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.281092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.281751 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.281755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.281865 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.282461 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:10.282478 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:10.286067 (+ 3589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":312,"threads_started":1}
W20260807 22:51:10.286486 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:10.277001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:47905 (local address 127.9.194.254:43971)
0807 22:51:10.277202 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:10.277209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:10.277650 (+ 441us) server_negotiation.cc:407] Connection header received
0807 22:51:10.277868 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:10.277873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:10.277956 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:10.278081 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:10.279054 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.279840 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.281206 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.281601 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.281719 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.286250 (+ 4531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:47905: BlockingRecv error: recv got EOF from 127.9.194.195:47905 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:10 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:10.496991 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:10.487585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:54189)
0807 22:51:10.488119 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:10.488154 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:10.488285 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:10.488686 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:10.488690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:10.489145 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:10.489332 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.489350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.490592 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.490597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.491886 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:10.491904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.492601 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.492612 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:10.492736 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.493324 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:10.493340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:10.496773 (+ 3433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":305,"threads_started":1}
W20260807 22:51:10.497196 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:10.487620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:54189 (local address 127.9.194.254:43971)
0807 22:51:10.487782 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:10.487787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:10.488244 (+ 457us) server_negotiation.cc:407] Connection header received
0807 22:51:10.488383 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:10.488387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:10.488463 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:10.488617 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:10.489461 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.490427 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.492029 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:10.492424 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:10.492564 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:10.496943 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:54189: BlockingRecv error: recv got EOF from 127.9.194.194:54189 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:11.183485 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:11.174379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:37157)
0807 22:51:11.174847 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:11.174863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:11.174989 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:11.175491 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:11.175494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:11.175867 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:11.176042 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.176051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.177003 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.177006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.178168 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.178184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.179145 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.179150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.179266 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.179835 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:11.179852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:11.183185 (+ 3333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":253,"threads_started":1}
W20260807 22:51:11.183815 19960 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:11.174453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37157 (local address 127.9.194.254:43971)
0807 22:51:11.175221 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:11.175225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:11.175240 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:51:11.175282 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:11.175286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:11.175351 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:11.175463 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:11.176173 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.176887 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.178317 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.178976 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.179103 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.183545 (+ 4442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:37157: BlockingRecv error: recv got EOF from 127.9.194.193:37157 (error 108)
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":523,"threads_started":1}
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:11.296574 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:11.287310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:44295)
0807 22:51:11.287796 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:11.287812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:11.287919 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:11.288243 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:11.288246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:11.288708 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:11.288853 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.288861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.290035 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.290039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.291074 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.291085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.291923 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.291928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.292051 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.292635 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:11.292652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:11.296174 (+ 3522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":292,"threads_started":1}
W20260807 22:51:11.296813 19960 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:11.287334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:44295 (local address 127.9.194.254:43971)
0807 22:51:11.287564 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:11.287574 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:11.287891 (+ 317us) server_negotiation.cc:407] Connection header received
0807 22:51:11.288029 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:11.288033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:11.288112 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:11.288188 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:11.289062 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.289911 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.291216 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.291773 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.291875 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.296430 (+ 4555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:44295: BlockingRecv error: recv got EOF from 127.9.194.195:44295 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:11 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:11.507323 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:11.497932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:59241)
0807 22:51:11.498441 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:11.498457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:11.498588 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:11.498936 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:11.498941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:11.499534 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:11.499753 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.499767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.500826 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.500832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.501826 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:11.501837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.502540 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.502546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:11.502720 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.503372 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:11.503390 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:11.506956 (+ 3566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":270,"threads_started":1}
W20260807 22:51:11.507413 19960 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:11.498050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59241 (local address 127.9.194.254:43971)
0807 22:51:11.498274 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:11.498281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:11.498554 (+ 273us) server_negotiation.cc:407] Connection header received
0807 22:51:11.498686 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:11.498690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:11.498770 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:11.498899 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:11.499922 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.500678 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.501978 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:11.502379 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:11.502507 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:11.507161 (+ 4654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:59241: BlockingRecv error: recv got EOF from 127.9.194.194:59241 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:12.194170 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:12.184559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:55493)
0807 22:51:12.185118 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:12.185139 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:12.185265 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:12.185673 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:12.185681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:12.186281 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:12.186584 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.186598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.187804 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.187808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.188826 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.188838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.189608 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.189612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.189733 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.190320 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:12.190336 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:12.193803 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":303,"threads_started":1}
W20260807 22:51:12.194315 19965 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:12.184717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:55493 (local address 127.9.194.254:43971)
0807 22:51:12.185310 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:12.185317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:12.185338 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:51:12.185395 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:12.185402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:12.185498 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:12.185594 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:12.186751 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.187690 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.188969 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.189479 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.189574 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.194033 (+ 4459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:55493: BlockingRecv error: recv got EOF from 127.9.194.193:55493 (error 108)
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":339,"threads_started":1}
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:12.306761 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:12.297545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:39767)
0807 22:51:12.298042 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:12.298058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:12.298212 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:12.298554 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:12.298561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:12.298955 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:12.299120 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.299128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.300276 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.300283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.301264 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.301276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.302023 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.302031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.302199 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.302874 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:12.302893 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:12.306518 (+ 3625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
W20260807 22:51:12.306881 19965 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:12.297585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:39767 (local address 127.9.194.254:43971)
0807 22:51:12.297803 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:12.297812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:12.298141 (+ 329us) server_negotiation.cc:407] Connection header received
0807 22:51:12.298349 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:12.298353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:12.298412 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:12.298510 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:12.299269 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.300127 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.301410 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.301842 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.301978 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.306689 (+ 4711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:39767: BlockingRecv error: recv got EOF from 127.9.194.195:39767 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:12 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:12.517180 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:12.508262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:42053)
0807 22:51:12.508824 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:12.508840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:12.508955 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:12.509252 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:12.509255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:12.509684 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:12.509870 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.509879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.510843 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.510847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.511833 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:12.511845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.512485 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.512488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:12.512606 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.513193 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:12.513210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:12.516817 (+ 3607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":344,"threads_started":1}
W20260807 22:51:12.517302 19965 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:12.508361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:42053 (local address 127.9.194.254:43971)
0807 22:51:12.508531 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:12.508536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:12.508912 (+ 376us) server_negotiation.cc:407] Connection header received
0807 22:51:12.509052 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:12.509055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:12.509126 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:12.509204 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:12.510001 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.510734 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.511955 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:12.512348 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:12.512434 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:12.517028 (+ 4594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:42053: BlockingRecv error: recv got EOF from 127.9.194.194:42053 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:13.205562 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:13.195193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:47911)
0807 22:51:13.195969 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:13.195991 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:13.196185 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:13.196555 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:13.196560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:13.197029 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:13.197218 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.197228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.198306 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.198311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.199518 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.199532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.200457 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.200464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.200678 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.201496 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:13.201515 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:13.205294 (+ 3779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":432,"threads_started":1}
W20260807 22:51:13.205806 19969 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:13.195327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:47911 (local address 127.9.194.254:43971)
0807 22:51:13.196057 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:13.196061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:13.196081 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:51:13.196333 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:13.196336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:13.196395 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:13.196523 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:13.197364 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.198176 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.199705 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.200293 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.200427 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.205504 (+ 5077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:47911: BlockingRecv error: recv got EOF from 127.9.194.193:47911 (error 108)
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":451,"threads_started":1}
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:13.316478 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:13.307696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:42397)
0807 22:51:13.308241 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:13.308258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:13.308369 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:13.308650 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:13.308653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:13.309049 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:13.309231 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.309239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.310319 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.310322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.311210 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.311220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.312018 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.312022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.312150 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.312740 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:13.312757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:13.316265 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":291,"threads_started":1}
W20260807 22:51:13.316776 19969 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:13.307859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:42397 (local address 127.9.194.254:43971)
0807 22:51:13.308058 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:13.308063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:13.308332 (+ 269us) server_negotiation.cc:407] Connection header received
0807 22:51:13.308451 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:13.308455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:13.308521 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:13.308609 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:13.309363 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.310197 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.311368 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.311885 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.311991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.316486 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:42397: BlockingRecv error: recv got EOF from 127.9.194.195:42397 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:13 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:13.527098 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:13.518272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:48269)
0807 22:51:13.518884 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:13.518900 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:13.519019 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:13.519342 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:13.519354 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:13.519824 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:13.520002 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.520011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.521016 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.521020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.521959 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:13.521971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.522635 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.522639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:13.522767 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.523465 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:13.523496 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:13.526869 (+ 3373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":332,"threads_started":1}
W20260807 22:51:13.527315 19969 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:13.518386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:48269 (local address 127.9.194.254:43971)
0807 22:51:13.518567 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:13.518572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:13.518976 (+ 404us) server_negotiation.cc:407] Connection header received
0807 22:51:13.519132 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:13.519135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:13.519197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:13.519281 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:13.520130 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.520866 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.522079 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:13.522477 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:13.522573 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:13.527049 (+ 4476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:48269: BlockingRecv error: recv got EOF from 127.9.194.194:48269 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:14.215725 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:14.206475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:59209)
0807 22:51:14.207010 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:14.207038 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:14.207145 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:14.207521 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:14.207525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:14.207961 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:14.208115 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.208124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.209136 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.209140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.210073 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.210084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.210757 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.210761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.210886 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.211646 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:14.211664 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:14.215368 (+ 3704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":306,"threads_started":1}
W20260807 22:51:14.215911 19973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:14.206688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:59209 (local address 127.9.194.254:43971)
0807 22:51:14.207172 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:14.207176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:14.207211 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:51:14.207254 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:14.207257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:14.207319 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:14.207400 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:14.208246 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.208999 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.210206 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.210637 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.210724 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.215607 (+ 4883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:59209: BlockingRecv error: recv got EOF from 127.9.194.193:59209 (error 108)
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":251,"threads_started":1}
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:14.326861 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:14.317560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:52909)
0807 22:51:14.318174 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:14.318191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:14.318356 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:14.318785 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:14.318793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:14.319381 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:14.319668 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.319681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.320814 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.320817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.321749 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.321759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.322374 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.322378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.322510 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.323176 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:14.323195 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:14.326623 (+ 3428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":360,"threads_started":1}
W20260807 22:51:14.327065 19973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:14.317655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:52909 (local address 127.9.194.254:43971)
0807 22:51:14.317841 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:14.317846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:14.318294 (+ 448us) server_negotiation.cc:407] Connection header received
0807 22:51:14.318529 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:14.318535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:14.318619 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:14.318725 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:14.319841 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.320680 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.321864 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.322256 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.322341 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.326813 (+ 4472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:52909: BlockingRecv error: recv got EOF from 127.9.194.195:52909 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:14 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:14.537194 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:14.528076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:52161)
0807 22:51:14.528647 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:14.528669 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:14.528828 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:14.529179 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:14.529184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:14.529650 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:14.529794 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.529802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.530858 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.530864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.531787 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:14.531798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.532510 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.532515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:14.532708 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.533311 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:14.533328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:14.536979 (+ 3651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":320,"threads_started":1}
W20260807 22:51:14.537393 19973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:14.528359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:52161 (local address 127.9.194.254:43971)
0807 22:51:14.528600 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:14.528606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:14.528753 (+ 147us) server_negotiation.cc:407] Connection header received
0807 22:51:14.528955 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:14.528960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:14.529040 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:14.529144 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:14.529948 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.530716 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.531937 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:14.532354 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:14.532498 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:14.537153 (+ 4655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:52161: BlockingRecv error: recv got EOF from 127.9.194.194:52161 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.193@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:15.234035 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:15.217008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.193:55063)
0807 22:51:15.217548 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:15.217565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:15.217667 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:15.218000 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:15.218003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:15.218431 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:15.218668 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.218678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.220119 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.220123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.221037 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.221048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.221728 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.221732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.221863 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.229366 (+ 7503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:15.229387 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:15.233670 (+ 4283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":279,"threads_started":1}
W20260807 22:51:15.234155 19977 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:15.217120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:55063 (local address 127.9.194.254:43971)
0807 22:51:15.217643 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:15.217647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:15.217665 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:51:15.217770 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:15.217774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:15.217832 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:15.217921 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:15.218826 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.219972 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.221169 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.221597 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.221688 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.233905 (+ 12217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.193:55063: BlockingRecv error: recv got EOF from 127.9.194.193:55063 (error 108)
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":293,"threads_started":1}
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.195@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:15.336974 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:15.328030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.195:44579)
0807 22:51:15.328565 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:15.328584 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:15.328711 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:15.329056 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:15.329060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:15.329478 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:15.329633 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.329642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.330703 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.330707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.331641 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.331653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.332345 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.332349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.332480 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.333044 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:15.333062 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:15.336621 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":297,"threads_started":1}
W20260807 22:51:15.337083 19977 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:15.328102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:44579 (local address 127.9.194.254:43971)
0807 22:51:15.328419 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:15.328426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:15.328698 (+ 272us) server_negotiation.cc:407] Connection header received
0807 22:51:15.328829 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:15.328832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:15.328896 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:15.328972 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:15.329781 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.330564 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.331770 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.332190 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.332296 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.336831 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.195:44579: BlockingRecv error: recv got EOF from 127.9.194.195:44579 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:51:15 dist-test-slave-5k54 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.194.194@KRBTEST.COM for oryx/127.9.194.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:51:15.547992 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:15.538587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43971 (local address 127.9.194.194:46513)
0807 22:51:15.539147 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:15.539164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:15.539316 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:15.539652 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:15.539655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:15.540033 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:15.540210 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.540218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.541273 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.541280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.542528 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:15.542541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.543239 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.543248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:15.543468 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.544044 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:15.544062 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:15.547625 (+ 3563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.194.254:43971: Server oryx/127.9.194.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":274,"threads_started":1}
W20260807 22:51:15.548110 19977 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:15.538831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:46513 (local address 127.9.194.254:43971)
0807 22:51:15.539039 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:15.539046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:15.539263 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:51:15.539446 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:15.539450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:15.539521 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:15.539598 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:15.540357 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.541151 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.542655 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.543062 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.543151 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.547819 (+ 4668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.194.194:46513: BlockingRecv error: recv got EOF from 127.9.194.194:46513 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:51:15.977078 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19473
I20260807 22:51:15.989674 19981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:15.978127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.9.194.193:34015)
0807 22:51:15.978808 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:15.978815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:15.978860 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:51:15.978926 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:15.978933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:15.979167 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:15.979330 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:15.980763 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.981822 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.983044 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:15.983682 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:15.983783 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:15.986251 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:15.986259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:15.986272 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:15.986308 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:15.989049 (+ 2741us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:51:15.989411 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52602: Request ticket server kudu/127.9.194.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":382,"threads_started":1}
W20260807 22:51:15.989789 19981 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52602: Request ticket server kudu/127.9.194.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:51:15.989779 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19473: Not authorized: Client connection negotiation failed: client connection to 127.9.194.193:34015: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:51:15.990052 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19473
I20260807 22:51:16.009054 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19635
I20260807 22:51:16.022485 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:16.010346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.9.194.194:39949)
0807 22:51:16.011010 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:16.011017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:16.011047 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:51:16.011115 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:16.011121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:16.011374 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:16.011640 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:16.013010 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.014097 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.015347 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.016031 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.016192 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:16.018764 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:16.018773 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:16.018786 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:16.018824 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:16.021681 (+ 2857us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:51:16.022104 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60734: Request ticket server kudu/127.9.194.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":355,"threads_started":1}
W20260807 22:51:16.022496 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19635: Not authorized: Client connection negotiation failed: client connection to 127.9.194.194:39949: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:51:16.022581 19982 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60734: Request ticket server kudu/127.9.194.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:51:16.022673 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19635
I20260807 22:51:16.038429 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19772
I20260807 22:51:16.051327 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:16.039700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.9.194.195:40925)
0807 22:51:16.040387 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:16.040395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:16.040419 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:51:16.040490 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:16.040498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:16.040722 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:16.040936 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:16.042264 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.043295 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.044589 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.045174 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.045323 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:16.047928 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:16.047937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:16.047950 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:16.048000 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:16.050623 (+ 2623us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:51:16.051001 (+ 378us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38018: Request ticket server kudu/127.9.194.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":397,"threads_started":1}
W20260807 22:51:16.051414 19983 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38018: Request ticket server kudu/127.9.194.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:51:16.051337 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19772: Not authorized: Client connection negotiation failed: client connection to 127.9.194.195:40925: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.194.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:51:16.051641 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19772
I20260807 22:51:16.070791 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19401
I20260807 22:51:16.072188 19466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:16.195037 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19401
2026-08-07T22:51:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21850 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T22:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:51:16Z Disabled control of system clock
I20260807 22:51:16.278223 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33587
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:51:16.628782 19988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:16.629076 19988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:16.629155 19988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:16.636229 19988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:51:16.636329 19988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:16.636368 19988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:16.636425 19988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:51:16.636469 19988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:51:16.645288 19988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:16.646509 19988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:16.648227 19988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:16.659998 19993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:16.660032 19996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:16.660069 19994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:16.660983 19988 server_base.cc:1034] running on GCE node
I20260807 22:51:16.661808 19988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:16.663120 19988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:16.664320 19988 hybrid_clock.cc:648] HybridClock initialized: now 1786143076664283 us; error 35 us; skew 500 ppm
I20260807 22:51:16.664680 19988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:16.666917 19988 webserver.cc:460] Webserver started at http://127.9.194.254:41129/ using document root <none> and password file <none>
I20260807 22:51:16.667658 19988 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:16.667773 19988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:16.668116 19988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:16.670967 19988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "91e2d3a725b34f84b69f3a623987adfb"
format_stamp: "Formatted at 2026-08-07 22:51:16 on dist-test-slave-5k54"
server_key: "8ae21756d034d86777a96aa0ac957e2e"
server_key_iv: "1bd415da7f81b9435ed1293a178616aa"
server_key_version: "encryptionkey@0"
I20260807 22:51:16.671804 19988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "91e2d3a725b34f84b69f3a623987adfb"
format_stamp: "Formatted at 2026-08-07 22:51:16 on dist-test-slave-5k54"
server_key: "8ae21756d034d86777a96aa0ac957e2e"
server_key_iv: "1bd415da7f81b9435ed1293a178616aa"
server_key_version: "encryptionkey@0"
I20260807 22:51:16.676810 19988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:51:16.680095 20002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:16.681310 19988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:51:16.681495 19988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "91e2d3a725b34f84b69f3a623987adfb"
format_stamp: "Formatted at 2026-08-07 22:51:16 on dist-test-slave-5k54"
server_key: "8ae21756d034d86777a96aa0ac957e2e"
server_key_iv: "1bd415da7f81b9435ed1293a178616aa"
server_key_version: "encryptionkey@0"
I20260807 22:51:16.681716 19988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:16.694352 19988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:16.695096 19988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:16.695399 19988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:16.713565 19988 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33587
I20260807 22:51:16.713584 20054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33587 every 8 connection(s)
I20260807 22:51:16.714887 19988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:51:16.718991 20055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:16.720444 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 19988
I20260807 22:51:16.720650 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:51:16.721313 9995 external_mini_cluster.cc:1428] Setting key a0c83d7cfa1ef24d5d83408a86bf5404
I20260807 22:51:16.730808 20055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb: Bootstrap starting.
I20260807 22:51:16.734350 20055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:16.735378 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:16.723554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.9.194.254:33587)
0807 22:51:16.724771 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:16.724781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:16.724834 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:51:16.726548 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:16.726568 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:16.726982 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:16.727298 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:16.727997 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.729262 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.731108 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:16.731814 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:16.731938 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:16.733226 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:16.733232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:16.733245 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:16.733752 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:16.733834 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:16.733854 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:16.733946 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:16.734176 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 22:51:16.734481 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":546,"threads_started":1}
I20260807 22:51:16.735478 20055 log.cc:824] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb: Log is configured to *not* fsync() on all Append() calls
I20260807 22:51:16.738399 20055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb: No bootstrap required, opened a new log
I20260807 22:51:16.745617 20055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } }
I20260807 22:51:16.746002 20055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:16.746119 20055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e2d3a725b34f84b69f3a623987adfb, State: Initialized, Role: FOLLOWER
I20260807 22:51:16.746634 20055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } }
I20260807 22:51:16.746850 20055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:51:16.747025 20055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:51:16.747210 20055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:16.749344 20055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } }
I20260807 22:51:16.749830 20055 leader_election.cc:302] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91e2d3a725b34f84b69f3a623987adfb; no voters:
I20260807 22:51:16.750391 20055 leader_election.cc:288] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:51:16.750516 20060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:51:16.751801 20060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [term 1 LEADER]: Becoming Leader. State: Replica: 91e2d3a725b34f84b69f3a623987adfb, State: Running, Role: LEADER
I20260807 22:51:16.752277 20060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } }
I20260807 22:51:16.752508 20055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:51:16.755165 20062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91e2d3a725b34f84b69f3a623987adfb. Latest consensus state: current_term: 1 leader_uuid: "91e2d3a725b34f84b69f3a623987adfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } } }
I20260807 22:51:16.755702 20062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:16.755228 20061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91e2d3a725b34f84b69f3a623987adfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2d3a725b34f84b69f3a623987adfb" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33587 } } }
I20260807 22:51:16.756141 20061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:16.759658 20071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:51:16.764835 20071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:51:16.773093 20071 catalog_manager.cc:1347] Generated new cluster ID: 249ef75a61ad4f3697a115bc7e0c7610
I20260807 22:51:16.773219 20071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:51:16.788669 20071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:51:16.789863 20071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:51:16.801054 20071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91e2d3a725b34f84b69f3a623987adfb: Generated new TSK 0
I20260807 22:51:16.801864 20071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:51:16.812134 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--builtin_ntp_servers=127.9.194.212:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 22:51:17.152655 20080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:17.152977 20080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:17.153126 20080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:17.160414 20080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:17.160554 20080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:17.160722 20080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:51:17.169564 20080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:17.170823 20080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:17.172684 20080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:17.185652 20085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:17.185690 20086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:17.185745 20088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:17.186596 20080 server_base.cc:1034] running on GCE node
I20260807 22:51:17.187175 20080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:17.188127 20080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:17.189350 20080 hybrid_clock.cc:648] HybridClock initialized: now 1786143077189302 us; error 62 us; skew 500 ppm
I20260807 22:51:17.189663 20080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:17.191939 20080 webserver.cc:460] Webserver started at http://127.9.194.193:33731/ using document root <none> and password file <none>
I20260807 22:51:17.192631 20080 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:17.192764 20080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:17.193091 20080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:17.195997 20080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "7fdc798df5e3427aa10f4011f2977966"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "714f3202183e84ac369baeb5afe2880d"
server_key_iv: "020d3aa6eed92fef69ea909e7a34cb18"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.196769 20080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "7fdc798df5e3427aa10f4011f2977966"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "714f3202183e84ac369baeb5afe2880d"
server_key_iv: "020d3aa6eed92fef69ea909e7a34cb18"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.201678 20080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:51:17.204960 20095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.206223 20080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:51:17.206390 20080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "7fdc798df5e3427aa10f4011f2977966"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "714f3202183e84ac369baeb5afe2880d"
server_key_iv: "020d3aa6eed92fef69ea909e7a34cb18"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.206609 20080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:17.258163 20080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:17.259107 20080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:17.259505 20080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:17.260718 20080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:17.262434 20080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:17.262524 20080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.262609 20080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:17.262663 20080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.296898 20080 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39885
I20260807 22:51:17.296998 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39885 every 8 connection(s)
I20260807 22:51:17.298259 20080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:51:17.303856 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20080
I20260807 22:51:17.304065 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:51:17.304702 9995 external_mini_cluster.cc:1428] Setting key 5b6518283214ae861cb1849f85c8a227
I20260807 22:51:17.307577 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--builtin_ntp_servers=127.9.194.212:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:51:17.311851 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:17.300867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:59889 (local address 127.9.194.254:33587)
0807 22:51:17.301342 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:17.301346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:17.302545 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 22:51:17.304524 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:17.304529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:17.304624 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:17.304743 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:17.305948 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.306991 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.308710 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.309501 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.309643 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:17.311097 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:17.311102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:17.311105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:17.311121 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:17.311161 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:17.311173 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:17.311245 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:17.311519 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 22:51:17.311596 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":264,"threads_started":1}
I20260807 22:51:17.313309 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:17.301516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33587 (local address 127.9.194.193:59889)
0807 22:51:17.302389 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:17.302431 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:17.304256 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:17.304987 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:17.304996 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:17.305080 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:17.305778 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:17.305798 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.307164 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.307167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:17.308551 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:17.308563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.309694 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.309699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:17.309952 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:17.310726 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:17.310732 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:17.310785 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:17.310889 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:17.311289 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:17.311301 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:17.311312 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:51:17.311507 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 22:51:17.311930 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
I20260807 22:51:17.314742 20211 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33587
I20260807 22:51:17.315037 20211 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:17.315802 20211 heartbeater.cc:511] Master 127.9.194.254:33587 requested a full tablet report, sending...
I20260807 22:51:17.317764 20019 ts_manager.cc:194] Registered new tserver with Master: 7fdc798df5e3427aa10f4011f2977966 (127.9.194.193:39885)
I20260807 22:51:17.319249 20019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.193:59889
W20260807 22:51:17.644964 20216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:17.645329 20216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:17.645505 20216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:17.652684 20216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:17.652822 20216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:17.653013 20216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:51:17.662694 20216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:17.664116 20216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:17.665947 20216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:17.679281 20222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:17.679395 20221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:17.679968 20224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:17.680069 20216 server_base.cc:1034] running on GCE node
I20260807 22:51:17.680745 20216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:17.681753 20216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:17.682989 20216 hybrid_clock.cc:648] HybridClock initialized: now 1786143077682942 us; error 58 us; skew 500 ppm
I20260807 22:51:17.683326 20216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:17.685709 20216 webserver.cc:460] Webserver started at http://127.9.194.194:41053/ using document root <none> and password file <none>
I20260807 22:51:17.686542 20216 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:17.686679 20216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:17.687022 20216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:17.689924 20216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "e4a9ac37cf774df6bc34e34fc3c9a1cb"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "1c4c3ad2c9c8fd40c38752b705d4d266"
server_key_iv: "3c3919b14ed5fd8023b5f8e177d3be55"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.690752 20216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "e4a9ac37cf774df6bc34e34fc3c9a1cb"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "1c4c3ad2c9c8fd40c38752b705d4d266"
server_key_iv: "3c3919b14ed5fd8023b5f8e177d3be55"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.695982 20216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:51:17.699321 20230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.700616 20216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:51:17.700790 20216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "e4a9ac37cf774df6bc34e34fc3c9a1cb"
format_stamp: "Formatted at 2026-08-07 22:51:17 on dist-test-slave-5k54"
server_key: "1c4c3ad2c9c8fd40c38752b705d4d266"
server_key_iv: "3c3919b14ed5fd8023b5f8e177d3be55"
server_key_version: "encryptionkey@0"
I20260807 22:51:17.701009 20216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:17.725968 20216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:17.727016 20216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:17.727346 20216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:17.728458 20216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:17.730240 20216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:17.730362 20216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.730475 20216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:17.730556 20216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:17.764406 20216 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:43823
I20260807 22:51:17.764498 20345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:43823 every 8 connection(s)
I20260807 22:51:17.765726 20216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:51:17.766074 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20216
I20260807 22:51:17.766263 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:51:17.766887 9995 external_mini_cluster.cc:1428] Setting key 366610f8e3e2d76ae9ad789d2ffef84c
I20260807 22:51:17.770139 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--builtin_ntp_servers=127.9.194.212:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:51:17.781095 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:17.768531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:34747 (local address 127.9.194.254:33587)
0807 22:51:17.768754 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:17.768762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:17.770382 (+ 1620us) server_negotiation.cc:407] Connection header received
0807 22:51:17.773333 (+ 2951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:17.773338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:17.773403 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:17.773494 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:17.774921 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.775738 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.778224 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.778602 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.778690 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:17.780301 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:17.780305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:17.780309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:17.780325 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:17.780356 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:17.780367 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:17.780435 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:17.780708 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 22:51:17.780830 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 22:51:17.782162 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:17.769102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33587 (local address 127.9.194.194:34747)
0807 22:51:17.770219 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:17.770270 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:17.773041 (+ 2771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:17.773743 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:17.773757 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:17.773902 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:17.774737 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:17.774763 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.775900 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.775908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:17.778086 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:17.778101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:17.778816 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:17.778825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:17.779167 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:17.779944 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:17.779950 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:17.780002 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:17.780106 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:17.780456 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:17.780469 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:17.780479 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:51:17.780659 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 22:51:17.781110 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":325,"threads_started":1}
I20260807 22:51:17.783761 20346 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33587
I20260807 22:51:17.784087 20346 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:17.785063 20346 heartbeater.cc:511] Master 127.9.194.254:33587 requested a full tablet report, sending...
I20260807 22:51:17.786629 20019 ts_manager.cc:194] Registered new tserver with Master: e4a9ac37cf774df6bc34e34fc3c9a1cb (127.9.194.194:43823)
I20260807 22:51:17.787508 20019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.194:34747
W20260807 22:51:18.107457 20349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:18.107795 20349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:18.107957 20349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:18.115247 20349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:18.115388 20349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:18.115582 20349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:51:18.124567 20349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:18.125880 20349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:18.127688 20349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:18.141255 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:18.141279 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:18.141439 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:18.142100 20349 server_base.cc:1034] running on GCE node
I20260807 22:51:18.142644 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:18.143604 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:18.144857 20349 hybrid_clock.cc:648] HybridClock initialized: now 1786143078144753 us; error 49 us; skew 500 ppm
I20260807 22:51:18.145128 20349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:18.147352 20349 webserver.cc:460] Webserver started at http://127.9.194.195:46643/ using document root <none> and password file <none>
I20260807 22:51:18.148089 20349 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:18.148180 20349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:18.148445 20349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:18.151117 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "e18166bea9134eec8657e8c43aa1715d"
format_stamp: "Formatted at 2026-08-07 22:51:18 on dist-test-slave-5k54"
server_key: "e520f0de6d91a8c67980900b22a45793"
server_key_iv: "f6e88692b31fbf0cd80212ab349e7c13"
server_key_version: "encryptionkey@0"
I20260807 22:51:18.151975 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "e18166bea9134eec8657e8c43aa1715d"
format_stamp: "Formatted at 2026-08-07 22:51:18 on dist-test-slave-5k54"
server_key: "e520f0de6d91a8c67980900b22a45793"
server_key_iv: "f6e88692b31fbf0cd80212ab349e7c13"
server_key_version: "encryptionkey@0"
I20260807 22:51:18.157003 20349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:51:18.160378 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:18.161624 20349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:51:18.161798 20349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "e18166bea9134eec8657e8c43aa1715d"
format_stamp: "Formatted at 2026-08-07 22:51:18 on dist-test-slave-5k54"
server_key: "e520f0de6d91a8c67980900b22a45793"
server_key_iv: "f6e88692b31fbf0cd80212ab349e7c13"
server_key_version: "encryptionkey@0"
I20260807 22:51:18.162014 20349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:18.185105 20349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:18.185932 20349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:18.186228 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:18.187161 20349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:18.188601 20349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:18.188714 20349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:18.188831 20349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:18.188925 20349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:18.222453 20349 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:46653
I20260807 22:51:18.222546 20479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:46653 every 8 connection(s)
I20260807 22:51:18.223819 20349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:51:18.231310 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20349
I20260807 22:51:18.231523 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:51:18.231993 9995 external_mini_cluster.cc:1428] Setting key cf0adaf447bb82ec53aaba21088e7db9
I20260807 22:51:18.237393 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:18.226560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:45427 (local address 127.9.194.254:33587)
0807 22:51:18.226777 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:18.226783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:18.228395 (+ 1612us) server_negotiation.cc:407] Connection header received
0807 22:51:18.230106 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:18.230110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:18.230175 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:18.230264 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:18.231638 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.232433 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.234521 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.235045 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.235183 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:18.236534 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:18.236539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:18.236542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:18.236558 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:18.236588 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:18.236600 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:18.236697 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:18.236930 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 22:51:18.237065 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:51:18.238359 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:18.227154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:33587 (local address 127.9.194.195:45427)
0807 22:51:18.228228 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:18.228271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:18.229870 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:18.230510 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:18.230519 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:18.230605 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:18.231395 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:18.231422 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.232591 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.232597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:18.234386 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:18.234400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.235269 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.235277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:18.235578 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:18.236211 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:18.236213 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:18.236251 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:18.236328 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:18.236698 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:18.236711 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:18.236721 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:51:18.236892 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 22:51:18.237281 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":374,"threads_started":1}
I20260807 22:51:18.239697 20480 heartbeater.cc:347] Connected to a master server at 127.9.194.254:33587
I20260807 22:51:18.240000 20480 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:18.240729 20480 heartbeater.cc:511] Master 127.9.194.254:33587 requested a full tablet report, sending...
I20260807 22:51:18.242066 20019 ts_manager.cc:194] Registered new tserver with Master: e18166bea9134eec8657e8c43aa1715d (127.9.194.195:46653)
I20260807 22:51:18.242805 20019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.195:45427
I20260807 22:51:18.247079 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:51:18.253368 20213 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:18.251761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.9.194.254:33587)
0807 22:51:18.251918 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:18.251922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:18.252514 (+ 592us) server_negotiation.cc:407] Connection header received
0807 22:51:18.252685 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:18.252688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:18.252757 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:18.252838 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:18.253213 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41698: BlockingRecv error: recv got EOF from 127.0.0.1:41698 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:51:18.254228 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20080
I20260807 22:51:18.261637 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:18.255108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.9.194.193:39885)
0807 22:51:18.255691 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:18.255703 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:18.255728 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:51:18.255782 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:18.255787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:18.255984 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:18.256104 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:18.256527 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.257588 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.258748 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.259262 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.259356 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:18.260304 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:18.260311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:18.260324 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:18.260933 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:18.260978 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:18.260997 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:18.261084 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:18.261237 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:51:18.261339 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":326,"threads_started":1}
I20260807 22:51:18.262847 20206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:18.346583 20211 heartbeater.cc:503] Master 127.9.194.254:33587 was elected leader, sending a full tablet report...
I20260807 22:51:18.348891 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20080
I20260807 22:51:18.369771 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20216
I20260807 22:51:18.377264 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:18.370680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.9.194.194:43823)
0807 22:51:18.371223 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:18.371230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:18.371258 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:51:18.371320 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:18.371326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:18.371602 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:18.371738 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:18.372106 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.373088 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.374287 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.374821 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.374933 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:18.375948 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:18.375956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:18.375969 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:18.376550 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:18.376601 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:18.376620 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:18.376709 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:18.376830 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 22:51:18.376971 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":290,"threads_started":1}
I20260807 22:51:18.378471 20341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:18.459750 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20216
I20260807 22:51:18.477588 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20349
I20260807 22:51:18.485654 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:18.478639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.9.194.195:46653)
0807 22:51:18.479213 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:18.479220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:18.479249 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:51:18.479307 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:18.479313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:18.479594 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:18.479728 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:18.480117 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.481130 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.482354 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:18.482910 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:18.483003 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:18.484065 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:18.484073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:18.484086 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:18.484703 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:18.484755 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:18.484776 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:18.484920 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:18.485087 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:51:18.485238 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":284,"threads_started":1}
I20260807 22:51:18.487116 20475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:18.570009 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20349
I20260807 22:51:18.588446 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 19988
I20260807 22:51:18.589671 20050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:18.693696 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 19988
2026-08-07T22:51:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2496 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T22:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:51:18Z Disabled control of system clock
I20260807 22:51:18.775733 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41517
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:51:19.110826 20501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:19.111117 20501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:19.111202 20501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:19.118434 20501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:51:19.118538 20501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:19.118587 20501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:19.118637 20501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:51:19.118681 20501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:51:19.127902 20501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:41517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:19.129196 20501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:19.130980 20501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:19.142952 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:19.142863 20509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:19.142913 20507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:19.144148 20501 server_base.cc:1034] running on GCE node
I20260807 22:51:19.144884 20501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:19.146198 20501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:19.147485 20501 hybrid_clock.cc:648] HybridClock initialized: now 1786143079147328 us; error 116 us; skew 500 ppm
I20260807 22:51:19.147850 20501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:19.150120 20501 webserver.cc:460] Webserver started at http://127.9.194.254:34989/ using document root <none> and password file <none>
I20260807 22:51:19.150833 20501 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:19.150964 20501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:19.151327 20501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:19.156111 20501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "994add7fb95645cc9bce18c307aa5331"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "dbba3b19e518e6388347e30a09894614"
server_key_iv: "8cf82d27d4904089454e5515f9c7920a"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.156961 20501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "994add7fb95645cc9bce18c307aa5331"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "dbba3b19e518e6388347e30a09894614"
server_key_iv: "8cf82d27d4904089454e5515f9c7920a"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.162073 20501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:51:19.165486 20515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:19.166766 20501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:51:19.166953 20501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "994add7fb95645cc9bce18c307aa5331"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "dbba3b19e518e6388347e30a09894614"
server_key_iv: "8cf82d27d4904089454e5515f9c7920a"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.167202 20501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:19.173105 20501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:19.173858 20501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:19.174168 20501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:19.192655 20501 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:41517
I20260807 22:51:19.192687 20568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:41517 every 8 connection(s)
I20260807 22:51:19.194005 20501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:51:19.197762 20569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:19.199703 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20501
I20260807 22:51:19.199934 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:51:19.200584 9995 external_mini_cluster.cc:1428] Setting key f1901133cf32cc12a96dc92023a36c3e
I20260807 22:51:19.209156 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:19.202712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58988 (local address 127.9.194.254:41517)
0807 22:51:19.203925 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:19.203936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:19.203979 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:51:19.205586 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:19.205611 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:19.206023 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:19.206363 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:19.207553 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:19.207559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:19.207571 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:19.207588 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:19.207655 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:19.207674 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:19.207860 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:19.207965 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 22:51:19.208243 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":488,"threads_started":1}
I20260807 22:51:19.209837 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331: Bootstrap starting.
I20260807 22:51:19.212929 20569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:19.213923 20569 log.cc:824] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331: Log is configured to *not* fsync() on all Append() calls
I20260807 22:51:19.216501 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331: No bootstrap required, opened a new log
I20260807 22:51:19.223361 20569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } }
I20260807 22:51:19.223785 20569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:19.223889 20569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 994add7fb95645cc9bce18c307aa5331, State: Initialized, Role: FOLLOWER
I20260807 22:51:19.224478 20569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } }
I20260807 22:51:19.224722 20569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:51:19.224874 20569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:51:19.225051 20569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:19.227272 20569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } }
I20260807 22:51:19.227790 20569 leader_election.cc:302] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 994add7fb95645cc9bce18c307aa5331; no voters:
I20260807 22:51:19.228363 20569 leader_election.cc:288] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:51:19.228495 20574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:51:19.229241 20574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [term 1 LEADER]: Becoming Leader. State: Replica: 994add7fb95645cc9bce18c307aa5331, State: Running, Role: LEADER
I20260807 22:51:19.229816 20574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } }
I20260807 22:51:19.230484 20569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:51:19.232626 20576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 994add7fb95645cc9bce18c307aa5331. Latest consensus state: current_term: 1 leader_uuid: "994add7fb95645cc9bce18c307aa5331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } } }
I20260807 22:51:19.233086 20576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:19.234313 20575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "994add7fb95645cc9bce18c307aa5331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994add7fb95645cc9bce18c307aa5331" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41517 } } }
I20260807 22:51:19.235985 20575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331 [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:19.237164 20584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:51:19.241559 20584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:51:19.249637 20584 catalog_manager.cc:1347] Generated new cluster ID: b2f354025a254b12bc752351a1524053
I20260807 22:51:19.249750 20584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:51:19.260525 20584 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:51:19.260994 20584 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:51:19.270435 20584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 994add7fb95645cc9bce18c307aa5331: Generated new TSK 0
I20260807 22:51:19.271363 20584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:51:19.277226 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--builtin_ntp_servers=127.9.194.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:51:19.610605 20596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:19.610868 20596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:19.611037 20596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:19.618283 20596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:19.618381 20596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:19.618552 20596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:51:19.627679 20596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:19.628927 20596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:19.630707 20596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:19.643870 20602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:19.643941 20603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:19.643990 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:19.645040 20596 server_base.cc:1034] running on GCE node
I20260807 22:51:19.645686 20596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:19.646675 20596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:19.647917 20596 hybrid_clock.cc:648] HybridClock initialized: now 1786143079647884 us; error 90 us; skew 500 ppm
I20260807 22:51:19.648245 20596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:19.650583 20596 webserver.cc:460] Webserver started at http://127.9.194.193:34001/ using document root <none> and password file <none>
I20260807 22:51:19.651263 20596 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:19.651489 20596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:19.651836 20596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:19.654654 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "dbac36b7e92442768dcc89d263f17835"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "61d6e9ea3b1b2308c116d9a59faa99b8"
server_key_iv: "5228ef4df2b9973303dd8f7f980b8584"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.655520 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "dbac36b7e92442768dcc89d263f17835"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "61d6e9ea3b1b2308c116d9a59faa99b8"
server_key_iv: "5228ef4df2b9973303dd8f7f980b8584"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.660676 20596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:51:19.664059 20612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:19.665524 20596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:51:19.665697 20596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "dbac36b7e92442768dcc89d263f17835"
format_stamp: "Formatted at 2026-08-07 22:51:19 on dist-test-slave-5k54"
server_key: "61d6e9ea3b1b2308c116d9a59faa99b8"
server_key_iv: "5228ef4df2b9973303dd8f7f980b8584"
server_key_version: "encryptionkey@0"
I20260807 22:51:19.665918 20596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:19.671862 20596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:19.672626 20596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:19.672886 20596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:19.673761 20596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:19.675154 20596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:19.675282 20596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:19.675405 20596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:19.675570 20596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 22:51:19.708645 20596 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:41849
I20260807 22:51:19.708667 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:41849 every 8 connection(s)
I20260807 22:51:19.709970 20596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:51:19.715077 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20596
I20260807 22:51:19.715297 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:51:19.716120 9995 external_mini_cluster.cc:1428] Setting key 4bfcc3c011310922eb3cf38fb580b392
I20260807 22:51:19.719055 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:19.712668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:38163 (local address 127.9.194.254:41517)
0807 22:51:19.713110 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:19.713115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:19.714190 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 22:51:19.716690 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:19.716695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:19.716787 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:19.716920 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:19.718388 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:19.718392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:19.718395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:19.718410 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:19.718452 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:19.718464 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:19.718563 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:19.718795 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 22:51:19.718844 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":235,"threads_started":1}
I20260807 22:51:19.719302 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--builtin_ntp_servers=127.9.194.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:51:19.720391 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:19.713142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41517 (local address 127.9.194.193:38163)
0807 22:51:19.714031 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:19.714076 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:19.716389 (+ 2313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:19.717185 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:19.717196 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:19.717317 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:19.718051 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:19.718057 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:19.718105 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:19.718220 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:19.718578 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:19.718589 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:19.718596 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:51:19.718874 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 22:51:19.719144 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":259,"threads_started":1}
I20260807 22:51:19.721796 20732 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41517
I20260807 22:51:19.722087 20732 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:19.722651 20732 heartbeater.cc:511] Master 127.9.194.254:41517 requested a full tablet report, sending...
I20260807 22:51:19.724581 20532 ts_manager.cc:194] Registered new tserver with Master: dbac36b7e92442768dcc89d263f17835 (127.9.194.193:41849)
W20260807 22:51:20.046811 20736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:20.047086 20736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:20.047257 20736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:20.054376 20736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:20.054466 20736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:20.054674 20736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:51:20.063477 20736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:20.064692 20736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:20.066411 20736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:20.079885 20745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:20.079885 20743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:20.079919 20742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:20.080746 20736 server_base.cc:1034] running on GCE node
I20260807 22:51:20.081346 20736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:20.082266 20736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:20.083540 20736 hybrid_clock.cc:648] HybridClock initialized: now 1786143080083479 us; error 73 us; skew 500 ppm
I20260807 22:51:20.083871 20736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:20.086179 20736 webserver.cc:460] Webserver started at http://127.9.194.194:43081/ using document root <none> and password file <none>
I20260807 22:51:20.086900 20736 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:20.087035 20736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:20.087375 20736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:20.090173 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "03701fc5baa74ee198516df830fc6d45"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "8e199afe791eeb3407b6260ba25d87fa"
server_key_iv: "df33f0734d3ada5082bce9c03bc9d196"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.090981 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "03701fc5baa74ee198516df830fc6d45"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "8e199afe791eeb3407b6260ba25d87fa"
server_key_iv: "df33f0734d3ada5082bce9c03bc9d196"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.096460 20736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 22:51:20.099965 20752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.101359 20736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:51:20.101495 20736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "03701fc5baa74ee198516df830fc6d45"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "8e199afe791eeb3407b6260ba25d87fa"
server_key_iv: "df33f0734d3ada5082bce9c03bc9d196"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.101670 20736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:20.107208 20736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:20.107986 20736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:20.108202 20736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:20.109035 20736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:20.110317 20736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:20.110402 20736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.110484 20736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:20.110539 20736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.143186 20736 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:37405
I20260807 22:51:20.143230 20867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:37405 every 8 connection(s)
I20260807 22:51:20.144534 20736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:51:20.146938 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20736
I20260807 22:51:20.147135 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:51:20.147689 9995 external_mini_cluster.cc:1428] Setting key a433b0d45334c11e2d9c0c218877add0
I20260807 22:51:20.150686 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--builtin_ntp_servers=127.9.194.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:51:20.153440 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.147128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:60633 (local address 127.9.194.254:41517)
0807 22:51:20.147322 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.147327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.149172 (+ 1845us) server_negotiation.cc:407] Connection header received
0807 22:51:20.151092 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.151100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.151167 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.151295 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.152631 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:20.152639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:20.152646 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:20.152668 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:20.152713 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:20.152731 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:20.152853 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:20.153015 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:51:20.153084 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:51:20.154243 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.147877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41517 (local address 127.9.194.194:60633)
0807 22:51:20.148998 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:20.149054 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:20.150771 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:20.151551 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:20.151560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:20.151647 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:20.152336 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:20.152338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:20.152374 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:20.152452 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:20.152828 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:20.152836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:20.152841 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:51:20.153046 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 22:51:20.153229 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":353,"threads_started":1}
I20260807 22:51:20.155669 20868 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41517
I20260807 22:51:20.155961 20868 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:20.156499 20868 heartbeater.cc:511] Master 127.9.194.254:41517 requested a full tablet report, sending...
I20260807 22:51:20.157766 20532 ts_manager.cc:194] Registered new tserver with Master: 03701fc5baa74ee198516df830fc6d45 (127.9.194.194:37405)
W20260807 22:51:20.483039 20873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:20.483314 20873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:20.483403 20873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:20.490731 20873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:20.490829 20873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:20.490943 20873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:51:20.499822 20873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:20.501086 20873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:20.502871 20873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:20.516259 20878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:20.516183 20879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:20.516366 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:20.517379 20873 server_base.cc:1034] running on GCE node
I20260807 22:51:20.517966 20873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:20.518875 20873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:20.520095 20873 hybrid_clock.cc:648] HybridClock initialized: now 1786143080520057 us; error 52 us; skew 500 ppm
I20260807 22:51:20.520411 20873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:51:20.525950 20873 webserver.cc:460] Webserver started at http://127.9.194.195:35347/ using document root <none> and password file <none>
I20260807 22:51:20.526695 20873 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:20.526827 20873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:20.527163 20873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:20.530002 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "cdbedd81662e498e99d233804f837fef"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "d95c054aae6059cd27c57e46b96d23cd"
server_key_iv: "93048ec3fcdaadf729c655620203ceac"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.530889 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "cdbedd81662e498e99d233804f837fef"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "d95c054aae6059cd27c57e46b96d23cd"
server_key_iv: "93048ec3fcdaadf729c655620203ceac"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.536378 20873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 22:51:20.539831 20889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.541083 20873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:51:20.541261 20873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "cdbedd81662e498e99d233804f837fef"
format_stamp: "Formatted at 2026-08-07 22:51:20 on dist-test-slave-5k54"
server_key: "d95c054aae6059cd27c57e46b96d23cd"
server_key_iv: "93048ec3fcdaadf729c655620203ceac"
server_key_version: "encryptionkey@0"
I20260807 22:51:20.541479 20873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:20.546936 20873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:20.547761 20873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:20.548032 20873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:20.548898 20873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:20.550237 20873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:20.550357 20873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.550493 20873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:20.550599 20873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:20.583300 20873 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:44759
I20260807 22:51:20.583356 21005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:44759 every 8 connection(s)
I20260807 22:51:20.584661 20873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:51:20.587471 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 20873
I20260807 22:51:20.587671 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:51:20.588212 9995 external_mini_cluster.cc:1428] Setting key f3762f60844a73e70def546c934709e7
I20260807 22:51:20.593338 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.587251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:39155 (local address 127.9.194.254:41517)
0807 22:51:20.587494 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.587498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.589285 (+ 1787us) server_negotiation.cc:407] Connection header received
0807 22:51:20.591198 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.591202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.591256 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.591368 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.592619 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:20.592622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:20.592625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:20.592641 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:20.592672 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:20.592682 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:20.592772 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:20.592911 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 22:51:20.592960 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:51:20.594396 21008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.588006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41517 (local address 127.9.194.195:39155)
0807 22:51:20.589132 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:20.589169 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:20.590917 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:20.591623 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:20.591631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:20.591721 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:20.592344 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:51:20.592346 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:20.592384 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:51:20.592469 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:20.592751 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:20.592756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:20.592761 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:51:20.592994 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 22:51:20.593259 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":350,"threads_started":1}
I20260807 22:51:20.595788 21006 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41517
I20260807 22:51:20.596081 21006 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:20.596647 21006 heartbeater.cc:511] Master 127.9.194.254:41517 requested a full tablet report, sending...
I20260807 22:51:20.597932 20532 ts_manager.cc:194] Registered new tserver with Master: cdbedd81662e498e99d233804f837fef (127.9.194.195:44759)
I20260807 22:51:20.603554 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:51:20.609617 20734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:51:20.608117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.9.194.254:41517)
0807 22:51:20.608282 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.608290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.608744 (+ 454us) server_negotiation.cc:407] Connection header received
0807 22:51:20.608927 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.608932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.608989 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.609084 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.609426 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59004: BlockingRecv error: recv got EOF from 127.0.0.1:59004 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:51:20.610625 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20596
I20260807 22:51:20.614147 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.611534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.9.194.193:41849)
0807 22:51:20.612043 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.612049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.612076 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:51:20.612128 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.612135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.612319 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.612489 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.613456 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:20.613462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:20.613472 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:20.613490 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:20.613527 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:20.613540 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:20.613684 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:20.613786 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 22:51:20.613882 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":214,"threads_started":1}
I20260807 22:51:20.615244 20726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:20.696841 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20596
I20260807 22:51:20.715749 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20736
I20260807 22:51:20.719300 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.716661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.9.194.194:37405)
0807 22:51:20.717187 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.717194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.717219 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:51:20.717292 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.717298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.717519 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.717639 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.718567 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:20.718574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:20.718586 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:20.718600 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:20.718651 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:20.718665 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:20.718765 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:20.718903 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:51:20.719026 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":257,"threads_started":1}
I20260807 22:51:20.720469 20863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:20.802250 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20736
I20260807 22:51:20.819464 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20873
I20260807 22:51:20.823663 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:20.820474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.9.194.195:44759)
0807 22:51:20.821136 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:20.821143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:20.821173 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:51:20.821241 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:20.821248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:20.821491 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:20.821667 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:20.822842 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:20.822848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:20.822859 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:51:20.822877 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:20.822923 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:51:20.822937 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:20.823124 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:20.823200 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 22:51:20.823305 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":332,"threads_started":1}
I20260807 22:51:20.824896 21000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:20.910032 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20873
I20260807 22:51:20.927933 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 20501
I20260807 22:51:20.929145 20563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:21.031956 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 20501
2026-08-07T22:51:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2337 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:51:21 dist-test-slave-5k54 krb5kdc[21025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:51:21 dist-test-slave-5k54 krb5kdc[21025](info): set up 2 sockets
Aug 07 22:51:21 dist-test-slave-5k54 krb5kdc[21025](info): commencing operation
krb5kdc: starting...
W20260807 22:51:22.181438 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:51:22 dist-test-slave-5k54 krb5kdc[21025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:51:22.451787 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38359
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:51:22.789929 21040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:22.790269 21040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:22.790453 21040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:22.797508 21040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:51:22.797649 21040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:22.797734 21040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:22.797822 21040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:51:22.797943 21040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:51:22.807848 21040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:22.809221 21040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:22.810604 21040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:22.822468 21045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:22.822480 21049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:22.822485 21046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:22.823483 21040 server_base.cc:1034] running on GCE node
I20260807 22:51:22.824296 21040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:22.825707 21040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:22.826980 21040 hybrid_clock.cc:648] HybridClock initialized: now 1786143082826931 us; error 49 us; skew 500 ppm
I20260807 22:51:22.827391 21040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:22 dist-test-slave-5k54 krb5kdc[21025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:22.830561 21040 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:51:22.832171 21040 webserver.cc:460] Webserver started at http://127.9.194.254:40245/ using document root <none> and password file <none>
I20260807 22:51:22.832877 21040 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:22.833032 21040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:22.833344 21040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:22.836447 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "60b909c422d24e8e88139269c4bcb39e"
format_stamp: "Formatted at 2026-08-07 22:51:22 on dist-test-slave-5k54"
server_key: "95d032f1f6d193cb1d929b2fe20319f7"
server_key_iv: "fde1946f6dfd6a89728cc5bca1206904"
server_key_version: "encryptionkey@0"
I20260807 22:51:22.837222 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "60b909c422d24e8e88139269c4bcb39e"
format_stamp: "Formatted at 2026-08-07 22:51:22 on dist-test-slave-5k54"
server_key: "95d032f1f6d193cb1d929b2fe20319f7"
server_key_iv: "fde1946f6dfd6a89728cc5bca1206904"
server_key_version: "encryptionkey@0"
I20260807 22:51:22.842654 21040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:51:22.846408 21056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:22.847883 21040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:51:22.848101 21040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "60b909c422d24e8e88139269c4bcb39e"
format_stamp: "Formatted at 2026-08-07 22:51:22 on dist-test-slave-5k54"
server_key: "95d032f1f6d193cb1d929b2fe20319f7"
server_key_iv: "fde1946f6dfd6a89728cc5bca1206904"
server_key_version: "encryptionkey@0"
I20260807 22:51:22.848318 21040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:22.863796 21040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:22.867741 21040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:22.868026 21040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:22.890472 21040 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:38359
I20260807 22:51:22.890496 21110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:38359 every 8 connection(s)
I20260807 22:51:22.891939 21040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:51:22.895589 21111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:22.898934 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 21040
I20260807 22:51:22.899196 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:51:22.900058 9995 external_mini_cluster.cc:1428] Setting key bffa18dbdcfbb9e137b8b105c82933dd
I20260807 22:51:22.908612 21111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: Bootstrap starting.
I20260807 22:51:22.912338 21111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:22.913640 21111 log.cc:824] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:51:22 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:51:22.916563 21111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: No bootstrap required, opened a new log
I20260807 22:51:22.923313 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:22.902097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.9.194.254:38359)
0807 22:51:22.902832 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:22.902842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:22.902882 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:51:22.904556 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:22.904577 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:22.904992 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:22.905301 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:22.907159 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:22.908569 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:22.910312 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:22.910994 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:22.911119 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:22.916061 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:22.916069 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:22.916085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:22.916118 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:22.919241 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:22.919987 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:22.919994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:22.920000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:22.920164 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:22.920581 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:22.920592 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:22.920595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:22.921133 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:22.921490 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:22.921901 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 22:51:22.922330 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":334,"threads_started":1}
I20260807 22:51:22.924541 21111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } }
I20260807 22:51:22.924934 21111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:22.925061 21111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b909c422d24e8e88139269c4bcb39e, State: Initialized, Role: FOLLOWER
I20260807 22:51:22.925684 21111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } }
I20260807 22:51:22.925931 21111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:51:22.926083 21111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:51:22.926270 21111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:22.928572 21111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } }
I20260807 22:51:22.929045 21111 leader_election.cc:302] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60b909c422d24e8e88139269c4bcb39e; no voters:
I20260807 22:51:22.929603 21111 leader_election.cc:288] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:51:22.929744 21116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:51:22.930557 21116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [term 1 LEADER]: Becoming Leader. State: Replica: 60b909c422d24e8e88139269c4bcb39e, State: Running, Role: LEADER
I20260807 22:51:22.931064 21116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } }
I20260807 22:51:22.931890 21111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:51:22.934000 21118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60b909c422d24e8e88139269c4bcb39e. Latest consensus state: current_term: 1 leader_uuid: "60b909c422d24e8e88139269c4bcb39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } } }
I20260807 22:51:22.933979 21117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60b909c422d24e8e88139269c4bcb39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b909c422d24e8e88139269c4bcb39e" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38359 } } }
I20260807 22:51:22.934382 21117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:22.934382 21118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e [sys.catalog]: This master's current role is: LEADER
I20260807 22:51:22.936676 21122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:51:22.943634 21122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:51:22.953899 21122 catalog_manager.cc:1347] Generated new cluster ID: 8a79af4b51544dc8a23ff6d5a2ce6d29
I20260807 22:51:22.954020 21122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:51:22.997323 21122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:51:22.998631 21122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:51:23.011082 21122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: Generated new TSK 0
I20260807 22:51:23.012020 21122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:51:23.083052 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:51:23.420711 21138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:23.420982 21138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:23.421087 21138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:23.428408 21138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:23.428522 21138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:23.428637 21138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:51:23.438799 21138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:23.440148 21138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:23.441466 21138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:23.454597 21143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:23.454625 21146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:23.454861 21144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:23.455452 21138 server_base.cc:1034] running on GCE node
I20260807 22:51:23.456097 21138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:23.457067 21138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:23.458297 21138 hybrid_clock.cc:648] HybridClock initialized: now 1786143083458228 us; error 64 us; skew 500 ppm
I20260807 22:51:23.458637 21138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:23 dist-test-slave-5k54 krb5kdc[21025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:23.461802 21138 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:51:23.463599 21138 webserver.cc:460] Webserver started at http://127.9.194.193:40077/ using document root <none> and password file <none>
I20260807 22:51:23.464282 21138 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:23.464433 21138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:23.464740 21138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:23.467706 21138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "413a4579549741febd92c6b0b30cbc8f"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "5b509debaf65b6dce6089a0adda0f1f9"
server_key_iv: "a34a61259a30255c9859bba3ee9edbb8"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.468501 21138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "413a4579549741febd92c6b0b30cbc8f"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "5b509debaf65b6dce6089a0adda0f1f9"
server_key_iv: "a34a61259a30255c9859bba3ee9edbb8"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.473613 21138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:51:23.477003 21153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:23.478302 21138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:51:23.478475 21138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "413a4579549741febd92c6b0b30cbc8f"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "5b509debaf65b6dce6089a0adda0f1f9"
server_key_iv: "a34a61259a30255c9859bba3ee9edbb8"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.478673 21138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:23.493772 21138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:23.497689 21138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:23.497973 21138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:23.498910 21138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:23.500284 21138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:23.500388 21138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:23.500540 21138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:23.500623 21138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:23.533819 21138 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:46525
I20260807 22:51:23.533910 21273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:46525 every 8 connection(s)
I20260807 22:51:23.535137 21138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:51:23.540385 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 21138
I20260807 22:51:23.540704 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:51:23.541607 9995 external_mini_cluster.cc:1428] Setting key 717ab7c1854f9cf6cc22b020f78adbd3
Aug 07 22:51:23 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:51:23.557941 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:23.538095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58061 (local address 127.9.194.254:38359)
0807 22:51:23.538620 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:23.538625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:23.539597 (+ 972us) server_negotiation.cc:407] Connection header received
0807 22:51:23.542422 (+ 2825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:23.542428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:23.542521 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:23.542644 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:23.544441 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:23.545683 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:23.547294 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:23.547787 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:23.547932 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:23.551535 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:23.551542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:23.551550 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:23.551594 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:23.554539 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:23.555347 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:23.555355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:23.555360 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:23.555552 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:23.555964 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:23.555972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:23.555977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:23.556361 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:23.556511 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:23.557167 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:51:23.557436 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":307,"threads_started":1}
I20260807 22:51:23.558537 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:23.538663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38359 (local address 127.9.194.193:58061)
0807 22:51:23.539372 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:23.539473 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:23.542059 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:23.542906 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:23.542915 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:23.543647 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:23.544265 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:23.544282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:23.545850 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:23.545854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:23.547128 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:23.547141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:23.548025 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:23.548033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:23.548282 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:23.548991 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:23.549013 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:23.551276 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:23.554696 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:23.554708 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:23.554729 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:23.555203 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:23.555662 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:23.555665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:23.555667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:23.555850 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:23.556489 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:23.556496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:23.556798 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 22:51:23.557030 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 22:51:23.557418 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":266,"threads_started":1}
I20260807 22:51:23.560251 21274 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38359
I20260807 22:51:23.560597 21274 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:23.561533 21274 heartbeater.cc:511] Master 127.9.194.254:38359 requested a full tablet report, sending...
I20260807 22:51:23.563931 21073 ts_manager.cc:194] Registered new tserver with Master: 413a4579549741febd92c6b0b30cbc8f (127.9.194.193:46525)
I20260807 22:51:23.565416 21073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:58061
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:51:23.606523 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:51:23.934445 21280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:23.934749 21280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:23.934921 21280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:23.942116 21280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:23.942267 21280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:23.942447 21280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:51:23.952212 21280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:23.953537 21280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:23.954852 21280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:23.968262 21285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:23.968194 21288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:23.968228 21286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:23.969101 21280 server_base.cc:1034] running on GCE node
I20260807 22:51:23.969681 21280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:23.970635 21280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:23.971825 21280 hybrid_clock.cc:648] HybridClock initialized: now 1786143083971773 us; error 48 us; skew 500 ppm
I20260807 22:51:23.972143 21280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:23 dist-test-slave-5k54 krb5kdc[21025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:23.975392 21280 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:51:23.976892 21280 webserver.cc:460] Webserver started at http://127.9.194.194:41769/ using document root <none> and password file <none>
I20260807 22:51:23.977586 21280 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:23.977720 21280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:23.978020 21280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:23.980964 21280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "aaf774390d16427990a06b1a94f15d26"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "d2e94700ce1675cf7b045d03d631215a"
server_key_iv: "418c38146a92956b2a2d84c08c50dfb5"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.981789 21280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "aaf774390d16427990a06b1a94f15d26"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "d2e94700ce1675cf7b045d03d631215a"
server_key_iv: "418c38146a92956b2a2d84c08c50dfb5"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.986981 21280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 22:51:23.990396 21296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:23.991729 21280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:51:23.991899 21280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "aaf774390d16427990a06b1a94f15d26"
format_stamp: "Formatted at 2026-08-07 22:51:23 on dist-test-slave-5k54"
server_key: "d2e94700ce1675cf7b045d03d631215a"
server_key_iv: "418c38146a92956b2a2d84c08c50dfb5"
server_key_version: "encryptionkey@0"
I20260807 22:51:23.992091 21280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:24.021189 21280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:24.025518 21280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:24.025815 21280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:24.026755 21280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:24.028129 21280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:24.028227 21280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:24.028380 21280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:24.028455 21280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:24.061843 21280 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:44887
I20260807 22:51:24.061893 21414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:44887 every 8 connection(s)
I20260807 22:51:24.063129 21280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:51:24.073151 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 21280
I20260807 22:51:24.073357 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:51:24.073947 9995 external_mini_cluster.cc:1428] Setting key f8c36d2ae43c5fe5512e7729fc1b0b70
Aug 07 22:51:24 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:51:24.083359 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.065751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:43603 (local address 127.9.194.254:38359)
0807 22:51:24.066224 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.066228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.067041 (+ 813us) server_negotiation.cc:407] Connection header received
0807 22:51:24.068906 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.068911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.069008 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.069127 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:24.070615 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.071345 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.072951 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.073475 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.073579 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.077518 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:24.077523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:24.077527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:24.077562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:24.080244 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.080947 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.080954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.080958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.081088 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.081525 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.081530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.081534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.081806 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:24.081960 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.082725 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.082961 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":268,"threads_started":1}
I20260807 22:51:24.084375 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.066226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38359 (local address 127.9.194.194:43603)
0807 22:51:24.066885 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.066931 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.068634 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.069364 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.069372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.069869 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:24.070444 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.070461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.071534 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.071538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.072793 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.072810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.073696 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.073703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.074013 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.074821 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:24.074850 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:24.077275 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:24.080382 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:24.080391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:24.080406 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:24.080827 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:24.081206 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:24.081211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:24.081215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:24.081406 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:24.081932 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:24.081940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:24.082358 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.082647 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.083181 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":266,"threads_started":1}
I20260807 22:51:24.085901 21415 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38359
I20260807 22:51:24.086239 21415 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:24.087139 21415 heartbeater.cc:511] Master 127.9.194.254:38359 requested a full tablet report, sending...
I20260807 22:51:24.088749 21073 ts_manager.cc:194] Registered new tserver with Master: aaf774390d16427990a06b1a94f15d26 (127.9.194.194:44887)
I20260807 22:51:24.089569 21073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:43603
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:51:24.135334 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:51:24.473191 21422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:51:24.473480 21422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:51:24.473577 21422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:51:24.481096 21422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:51:24.481213 21422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:51:24.481343 21422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:51:24.491410 21422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:51:24.492692 21422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:51:24.493983 21422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:51:24.507668 21430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:24.507746 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:51:24.507753 21432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:51:24.508426 21422 server_base.cc:1034] running on GCE node
I20260807 22:51:24.509025 21422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:51:24.510016 21422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:51:24.511240 21422 hybrid_clock.cc:648] HybridClock initialized: now 1786143084511198 us; error 51 us; skew 500 ppm
I20260807 22:51:24.511619 21422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:51:24 dist-test-slave-5k54 krb5kdc[21025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:51:24.514791 21422 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:51:24.516196 21422 webserver.cc:460] Webserver started at http://127.9.194.195:40201/ using document root <none> and password file <none>
I20260807 22:51:24.516886 21422 fs_manager.cc:359] Metadata directory not provided
I20260807 22:51:24.516988 21422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:51:24.517225 21422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:51:24.520100 21422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "410634a75e1f47a7adea47bf82f5de31"
format_stamp: "Formatted at 2026-08-07 22:51:24 on dist-test-slave-5k54"
server_key: "5239116ec0e1b4e2296a2f0aaeda1e98"
server_key_iv: "5aae7ae886647eb9b7905207fc43efdc"
server_key_version: "encryptionkey@0"
I20260807 22:51:24.520825 21422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "410634a75e1f47a7adea47bf82f5de31"
format_stamp: "Formatted at 2026-08-07 22:51:24 on dist-test-slave-5k54"
server_key: "5239116ec0e1b4e2296a2f0aaeda1e98"
server_key_iv: "5aae7ae886647eb9b7905207fc43efdc"
server_key_version: "encryptionkey@0"
I20260807 22:51:24.525858 21422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:51:24.529088 21439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:24.530328 21422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:51:24.530519 21422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "410634a75e1f47a7adea47bf82f5de31"
format_stamp: "Formatted at 2026-08-07 22:51:24 on dist-test-slave-5k54"
server_key: "5239116ec0e1b4e2296a2f0aaeda1e98"
server_key_iv: "5aae7ae886647eb9b7905207fc43efdc"
server_key_version: "encryptionkey@0"
I20260807 22:51:24.530717 21422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:51:24.553776 21422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:51:24.558290 21422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:51:24.558579 21422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:51:24.559624 21422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:51:24.560905 21422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:51:24.561017 21422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:24.561167 21422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:51:24.561241 21422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:51:24.569134 21274 heartbeater.cc:503] Master 127.9.194.254:38359 was elected leader, sending a full tablet report...
I20260807 22:51:24.594856 21422 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39439
I20260807 22:51:24.594903 21558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39439 every 8 connection(s)
I20260807 22:51:24.596202 21422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:51:24.606137 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 21422
I20260807 22:51:24.606344 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:51:24.606907 9995 external_mini_cluster.cc:1428] Setting key 78133b44eacb9ec80340052084f034b2
Aug 07 22:51:24 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:51:24.616089 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.598676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:46843 (local address 127.9.194.254:38359)
0807 22:51:24.599119 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.599126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.600351 (+ 1225us) server_negotiation.cc:407] Connection header received
0807 22:51:24.602243 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.602247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.602312 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.602405 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:24.604091 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.604892 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.606694 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.607329 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.607521 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.611044 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:24.611049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:24.611052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:24.611089 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:24.613466 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.614120 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.614123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.614125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.614250 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.614666 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.614669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.614671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.614907 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:24.615028 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.615673 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.615826 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":209,"threads_started":1}
I20260807 22:51:24.617010 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.599305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38359 (local address 127.9.194.195:46843)
0807 22:51:24.600180 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.600222 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.601984 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.602630 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.602639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.603250 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:51:24.603936 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.603955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.605047 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.605052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.606548 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.606561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.607643 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.607651 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.607901 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.608579 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:51:24.608601 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:51:24.610860 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:51:24.613592 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:24.613598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:24.613610 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:24.614015 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:24.614340 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:51:24.614343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:51:24.614346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:51:24.614560 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:51:24.615008 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:51:24.615014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:51:24.615305 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.615585 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.615953 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":355,"threads_started":1}
I20260807 22:51:24.618314 21559 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38359
I20260807 22:51:24.618626 21559 heartbeater.cc:464] Registering TS with master...
I20260807 22:51:24.619308 21559 heartbeater.cc:511] Master 127.9.194.254:38359 requested a full tablet report, sending...
I20260807 22:51:24.620826 21073 ts_manager.cc:194] Registered new tserver with Master: 410634a75e1f47a7adea47bf82f5de31 (127.9.194.195:39439)
I20260807 22:51:24.621726 21073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:46843
I20260807 22:51:24.622130 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:51:24.639974 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.626573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.9.194.254:38359)
0807 22:51:24.626761 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.626767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.627208 (+ 441us) server_negotiation.cc:407] Connection header received
0807 22:51:24.627332 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.627336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.627392 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.627572 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:24.628952 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.629677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.630904 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.631317 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.631482 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.633725 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:24.633730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:24.633733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:24.633762 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:24.636970 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.637710 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.637725 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.637728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.637881 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:24.638324 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:24.638330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:24.638335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:24.638686 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:24.638972 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.639364 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.639634 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:51:24.644052 21073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:51:24.650900 21073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:51:24.673965 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.665200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.9.194.194:44887)
0807 22:51:24.666318 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.666327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.666351 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:51:24.666443 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.666451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.666698 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.666899 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:51:24.667569 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.669058 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.671472 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.672780 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.672945 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.673144 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.673530 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.673703 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":800,"threads_started":1}
I20260807 22:51:24.674170 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.662841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39439 (local address 127.0.0.1:57106)
0807 22:51:24.664428 (+ 1587us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.664482 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.664679 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.665852 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.665862 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.665904 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:51:24.666339 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.666350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.668167 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.668174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.670854 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.670870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.672907 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.672916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.673332 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.673462 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.673551 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.673734 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":211,"thread_start_us":1177,"threads_started":1}
I20260807 22:51:24.674189 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.664289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.9.194.193:46525)
0807 22:51:24.664975 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.664998 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.665554 (+ 556us) server_negotiation.cc:407] Connection header received
0807 22:51:24.665852 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.665860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.666125 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.666304 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:51:24.667250 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.668817 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.670851 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.672376 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.672504 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.672738 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.673683 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.673844 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":325,"threads_started":1}
I20260807 22:51:24.674772 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.662989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57106 (local address 127.9.194.195:39439)
0807 22:51:24.664085 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.664094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.664891 (+ 797us) server_negotiation.cc:407] Connection header received
0807 22:51:24.665062 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.665073 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.665330 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.665534 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:51:24.666531 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.667965 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.671020 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.672374 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.672514 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.672739 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.674400 (+ 1661us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.674547 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":688,"threads_started":1}
I20260807 22:51:24.674894 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.663691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46525 (local address 127.0.0.1:56052)
0807 22:51:24.665433 (+ 1742us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.665456 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.665658 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.666588 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.666592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.666614 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:51:24.667041 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.667058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.668977 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.668985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.670688 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.670699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.672954 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.672961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.673359 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.673462 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.673551 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.674489 (+ 938us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1600,"mutex_wait_us":956,"thread_start_us":1736,"threads_started":1}
I20260807 22:51:24.675467 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.664767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44887 (local address 127.0.0.1:47314)
0807 22:51:24.665687 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.665704 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.665796 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.666923 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.666928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.666954 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:51:24.667374 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.667385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.669207 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.669213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.671283 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.671293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.673021 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.673031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.673282 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.673347 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.673485 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.675070 (+ 1585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":1443,"thread_start_us":369,"threads_started":1}
I20260807 22:51:24.678905 21346 tablet_service.cc:1467] Processing CreateTablet for tablet 155adb44c35449be97b016ea1dd0c54d (DEFAULT_TABLE table=test-table [id=c53107e970ad45389766ad77507d7776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:51:24.679087 21207 tablet_service.cc:1467] Processing CreateTablet for tablet 155adb44c35449be97b016ea1dd0c54d (DEFAULT_TABLE table=test-table [id=c53107e970ad45389766ad77507d7776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:51:24.679087 21490 tablet_service.cc:1467] Processing CreateTablet for tablet 155adb44c35449be97b016ea1dd0c54d (DEFAULT_TABLE table=test-table [id=c53107e970ad45389766ad77507d7776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:51:24.680217 21346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155adb44c35449be97b016ea1dd0c54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:24.680485 21207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155adb44c35449be97b016ea1dd0c54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:24.680485 21490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155adb44c35449be97b016ea1dd0c54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:51:24.691911 21579 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Bootstrap starting.
I20260807 22:51:24.695422 21579 tablet_bootstrap.cc:654] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:24.696548 21579 log.cc:824] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Log is configured to *not* fsync() on all Append() calls
I20260807 22:51:24.698045 21581 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Bootstrap starting.
I20260807 22:51:24.700174 21579 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: No bootstrap required, opened a new log
I20260807 22:51:24.700451 21579 ts_tablet_manager.cc:1397] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 22:51:24.701958 21581 tablet_bootstrap.cc:654] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:24.703305 21581 log.cc:824] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Log is configured to *not* fsync() on all Append() calls
I20260807 22:51:24.704483 21580 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Bootstrap starting.
I20260807 22:51:24.707981 21580 tablet_bootstrap.cc:654] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Neither blocks nor log segments found. Creating new log.
I20260807 22:51:24.709192 21580 log.cc:824] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:51:24.709797 21581 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: No bootstrap required, opened a new log
I20260807 22:51:24.710100 21581 ts_tablet_manager.cc:1397] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260807 22:51:24.711601 21579 raft_consensus.cc:348] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.712080 21579 raft_consensus.cc:374] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:24.712191 21579 raft_consensus.cc:729] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaf774390d16427990a06b1a94f15d26, State: Initialized, Role: FOLLOWER
I20260807 22:51:24.712423 21580 tablet_bootstrap.cc:492] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: No bootstrap required, opened a new log
I20260807 22:51:24.712677 21580 ts_tablet_manager.cc:1397] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 22:51:24.712841 21579 consensus_queue.cc:260] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.717059 21415 heartbeater.cc:503] Master 127.9.194.254:38359 was elected leader, sending a full tablet report...
I20260807 22:51:24.717762 21579 ts_tablet_manager.cc:1428] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 22:51:24.718823 21581 raft_consensus.cc:348] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.719182 21581 raft_consensus.cc:374] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:24.719306 21581 raft_consensus.cc:729] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410634a75e1f47a7adea47bf82f5de31, State: Initialized, Role: FOLLOWER
I20260807 22:51:24.719892 21581 consensus_queue.cc:260] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.720959 21559 heartbeater.cc:503] Master 127.9.194.254:38359 was elected leader, sending a full tablet report...
I20260807 22:51:24.722720 21581 ts_tablet_manager.cc:1428] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260807 22:51:24.723760 21580 raft_consensus.cc:348] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.724234 21580 raft_consensus.cc:374] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:51:24.724355 21580 raft_consensus.cc:729] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413a4579549741febd92c6b0b30cbc8f, State: Initialized, Role: FOLLOWER
I20260807 22:51:24.724961 21580 consensus_queue.cc:260] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.726635 21580 ts_tablet_manager.cc:1428] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260807 22:51:24.787719 21275 tablet.cc:2369] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:51:24.815160 21416 tablet.cc:2369] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:51:24.847934 21560 tablet.cc:2369] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:51:24.908900 21586 raft_consensus.cc:482] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:51:24.909214 21586 raft_consensus.cc:504] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.911039 21586 leader_election.cc:288] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaf774390d16427990a06b1a94f15d26 (127.9.194.194:44887), 413a4579549741febd92c6b0b30cbc8f (127.9.194.193:46525)
I20260807 22:51:24.917695 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.911561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44887 (local address 127.9.194.195:42269)
0807 22:51:24.911927 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.911956 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.912126 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.912576 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.912579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.912627 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:51:24.912841 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.912852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.914021 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.914026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.915591 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.915603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.916976 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.916983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.917178 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.917213 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.917315 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.917468 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":2}
I20260807 22:51:24.917762 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.911524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:42269 (local address 127.9.194.194:44887)
0807 22:51:24.911744 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.911750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.912062 (+ 312us) server_negotiation.cc:407] Connection header received
0807 22:51:24.912299 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.912305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.912387 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.912515 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:51:24.912981 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.913886 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.915743 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.916807 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.916920 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.916970 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.917356 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.917506 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:51:24.918184 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.911624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:39085 (local address 127.9.194.193:46525)
0807 22:51:24.911826 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.911840 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.912506 (+ 666us) server_negotiation.cc:407] Connection header received
0807 22:51:24.912675 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.912679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.912754 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.912856 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:51:24.913471 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.914359 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.916291 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.917219 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.917374 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.917440 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.917796 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.917944 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:51:24.918542 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.911576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46525 (local address 127.9.194.195:39085)
0807 22:51:24.912395 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 22:51:24.912412 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:51:24.912534 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:51:24.912876 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:51:24.912879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:51:24.912897 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:51:24.913310 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.913322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.914511 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.914515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.916133 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:51:24.916147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.917439 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.917446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:51:24.917616 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.917652 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:51:24.917752 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:51:24.918191 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":32,"thread_start_us":457,"threads_started":1}
I20260807 22:51:24.918859 21366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155adb44c35449be97b016ea1dd0c54d" candidate_uuid: "410634a75e1f47a7adea47bf82f5de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaf774390d16427990a06b1a94f15d26" is_pre_election: true
I20260807 22:51:24.919415 21366 raft_consensus.cc:2454] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410634a75e1f47a7adea47bf82f5de31 in term 0.
I20260807 22:51:24.919616 21227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155adb44c35449be97b016ea1dd0c54d" candidate_uuid: "410634a75e1f47a7adea47bf82f5de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413a4579549741febd92c6b0b30cbc8f" is_pre_election: true
I20260807 22:51:24.920089 21227 raft_consensus.cc:2454] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410634a75e1f47a7adea47bf82f5de31 in term 0.
I20260807 22:51:24.920167 21442 leader_election.cc:302] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410634a75e1f47a7adea47bf82f5de31, aaf774390d16427990a06b1a94f15d26; no voters:
I20260807 22:51:24.920606 21586 raft_consensus.cc:2781] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:51:24.920789 21586 raft_consensus.cc:482] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:51:24.920919 21586 raft_consensus.cc:3037] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:24.923120 21586 raft_consensus.cc:504] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.923770 21586 leader_election.cc:288] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [CANDIDATE]: Term 1 election: Requested vote from peers aaf774390d16427990a06b1a94f15d26 (127.9.194.194:44887), 413a4579549741febd92c6b0b30cbc8f (127.9.194.193:46525)
I20260807 22:51:24.924261 21227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155adb44c35449be97b016ea1dd0c54d" candidate_uuid: "410634a75e1f47a7adea47bf82f5de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413a4579549741febd92c6b0b30cbc8f"
I20260807 22:51:24.924335 21366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155adb44c35449be97b016ea1dd0c54d" candidate_uuid: "410634a75e1f47a7adea47bf82f5de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaf774390d16427990a06b1a94f15d26"
I20260807 22:51:24.924458 21227 raft_consensus.cc:3037] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:24.924537 21366 raft_consensus.cc:3037] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:51:24.926720 21366 raft_consensus.cc:2454] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410634a75e1f47a7adea47bf82f5de31 in term 1.
I20260807 22:51:24.926977 21227 raft_consensus.cc:2454] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410634a75e1f47a7adea47bf82f5de31 in term 1.
I20260807 22:51:24.927124 21442 leader_election.cc:302] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410634a75e1f47a7adea47bf82f5de31, aaf774390d16427990a06b1a94f15d26; no voters:
I20260807 22:51:24.927405 21586 raft_consensus.cc:2781] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:51:24.927846 21586 raft_consensus.cc:686] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 1 LEADER]: Becoming Leader. State: Replica: 410634a75e1f47a7adea47bf82f5de31, State: Running, Role: LEADER
I20260807 22:51:24.928382 21586 consensus_queue.cc:237] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } }
I20260807 22:51:24.932973 21072 catalog_manager.cc:5495] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 410634a75e1f47a7adea47bf82f5de31 (127.9.194.195). New cstate: current_term: 1 leader_uuid: "410634a75e1f47a7adea47bf82f5de31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "410634a75e1f47a7adea47bf82f5de31" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 } health_report { overall_health: UNKNOWN } } }
I20260807 22:51:24.970196 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:24.964718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.9.194.195:39439)
0807 22:51:24.964938 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:24.964946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:24.964966 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:51:24.965220 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:24.965224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:24.965314 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:24.965469 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:51:24.965840 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.966582 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.967983 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:24.968409 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:24.968548 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:24.969040 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:51:24.969570 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:51:24.969743 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:24.969837 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 22:51:24.969917 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:51:24.978603 21366 raft_consensus.cc:1261] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 1 FOLLOWER]: Refusing update from remote peer 410634a75e1f47a7adea47bf82f5de31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:51:24.978610 21227 raft_consensus.cc:1261] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 1 FOLLOWER]: Refusing update from remote peer 410634a75e1f47a7adea47bf82f5de31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:51:24.979640 21589 consensus_queue.cc:1035] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413a4579549741febd92c6b0b30cbc8f" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:51:24.980012 21586 consensus_queue.cc:1035] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaf774390d16427990a06b1a94f15d26" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:51:25.008474 21072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:51:25.008797 21072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:51:25.012580 21072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60b909c422d24e8e88139269c4bcb39e: Sending DeleteTablet for 3 replicas of tablet 155adb44c35449be97b016ea1dd0c54d
I20260807 22:51:25.013706 21490 tablet_service.cc:1514] Processing DeleteTablet for tablet 155adb44c35449be97b016ea1dd0c54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:51:25 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:57106
I20260807 22:51:25.013918 21346 tablet_service.cc:1514] Processing DeleteTablet for tablet 155adb44c35449be97b016ea1dd0c54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:51:25 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47314
I20260807 22:51:25.014533 21207 tablet_service.cc:1514] Processing DeleteTablet for tablet 155adb44c35449be97b016ea1dd0c54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:51:25 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:56052
I20260807 22:51:25.015204 21605 tablet_replica.cc:330] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: stopping tablet replica
I20260807 22:51:25.015704 21605 raft_consensus.cc:2229] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:51:25.015849 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 21138
I20260807 22:51:25.016549 21605 raft_consensus.cc:2258] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:51:25.017388 21607 tablet_replica.cc:330] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: stopping tablet replica
I20260807 22:51:25.017796 21607 raft_consensus.cc:2229] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:51:25.018289 21607 raft_consensus.cc:2258] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:51:25.020427 21607 ts_tablet_manager.cc:1905] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:51:25.020437 21605 ts_tablet_manager.cc:1905] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:51:25.027525 21607 ts_tablet_manager.cc:1918] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:51:25.027585 21605 ts_tablet_manager.cc:1918] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:51:25.027748 21605 log.cc:1196] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/155adb44c35449be97b016ea1dd0c54d
I20260807 22:51:25.027863 21607 log.cc:1196] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/155adb44c35449be97b016ea1dd0c54d
I20260807 22:51:25.028465 21607 ts_tablet_manager.cc:1939] T 155adb44c35449be97b016ea1dd0c54d P 413a4579549741febd92c6b0b30cbc8f: Deleting consensus metadata
I20260807 22:51:25.028590 21605 ts_tablet_manager.cc:1939] T 155adb44c35449be97b016ea1dd0c54d P 410634a75e1f47a7adea47bf82f5de31: Deleting consensus metadata
I20260807 22:51:25.030301 21059 catalog_manager.cc:4841] TS 410634a75e1f47a7adea47bf82f5de31 (127.9.194.195:39439): tablet 155adb44c35449be97b016ea1dd0c54d (table test-table [id=c53107e970ad45389766ad77507d7776]) successfully deleted
I20260807 22:51:25.030645 21057 catalog_manager.cc:4841] TS 413a4579549741febd92c6b0b30cbc8f (127.9.194.193:46525): tablet 155adb44c35449be97b016ea1dd0c54d (table test-table [id=c53107e970ad45389766ad77507d7776]) successfully deleted
I20260807 22:51:25.032698 21606 tablet_replica.cc:330] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: stopping tablet replica
I20260807 22:51:25.033082 21606 raft_consensus.cc:2229] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:51:25.033563 21606 raft_consensus.cc:2258] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:51:25.035755 21606 ts_tablet_manager.cc:1905] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:51:25.041095 21606 ts_tablet_manager.cc:1918] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:51:25.041272 21606 log.cc:1196] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/155adb44c35449be97b016ea1dd0c54d
I20260807 22:51:25.041718 21606 ts_tablet_manager.cc:1939] T 155adb44c35449be97b016ea1dd0c54d P aaf774390d16427990a06b1a94f15d26: Deleting consensus metadata
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:51:25.043241 21059 catalog_manager.cc:4841] TS aaf774390d16427990a06b1a94f15d26 (127.9.194.194:44887): tablet 155adb44c35449be97b016ea1dd0c54d (table test-table [id=c53107e970ad45389766ad77507d7776]) successfully deleted
I20260807 22:51:25.049198 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:25.017822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.9.194.193:46525)
0807 22:51:25.018022 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:25.018030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:25.031943 (+ 13913us) server_negotiation.cc:407] Connection header received
0807 22:51:25.032164 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:25.032172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:25.032266 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:25.032365 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:25.034259 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.035447 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.037262 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.037953 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.038083 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:25.043588 (+ 5505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:25.043600 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:25.043616 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:25.043652 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:25.046400 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.047217 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.047224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.047226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.047355 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.047779 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.047782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.047784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.048115 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:25.048345 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:25.048689 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 22:51:25.048845 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:51:25.049836 21269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:25.162177 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 21138
I20260807 22:51:25.181643 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 21280
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:51:25.199261 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:25.182564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.9.194.194:44887)
0807 22:51:25.182756 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:25.182761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:25.182806 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:51:25.183029 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:25.183033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:25.183096 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:25.183181 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:25.184755 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.185526 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.187325 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.187813 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.187911 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:25.192976 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:25.192998 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:25.193014 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:25.193052 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:25.196366 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.197179 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.197195 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.197197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.197321 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.197700 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.197704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.197706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.198078 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:25.198386 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:25.198699 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 22:51:25.198917 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:51:25.199965 21410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:25.297925 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 21280
I20260807 22:51:25.318209 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 21422
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:51:25.335888 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:51:25.319022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.9.194.195:39439)
0807 22:51:25.319230 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:51:25.319236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:51:25.319259 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:51:25.319405 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:51:25.319411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:51:25.319548 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:51:25.319672 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:51:25.320924 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.322165 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.323356 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:51:25.324171 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:51:25.324319 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:51:25.328747 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:51:25.328766 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:51:25.328791 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:51:25.328833 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:51:25.332893 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.333638 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.333653 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.333659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.333859 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:51:25.334245 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:51:25.334250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:51:25.334254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:51:25.334616 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:51:25.334965 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 22:51:25.335215 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:51:25.335486 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:51:25.336615 21554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:25.465260 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 21422
I20260807 22:51:25.484534 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 21040
I20260807 22:51:25.485766 21106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:51:25.597121 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 21040
2026-08-07T22:51:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4567 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21615](info): set up 2 sockets
Aug 07 22:51:25 dist-test-slave-5k54 krb5kdc[21615](info): commencing operation
krb5kdc: starting...
W20260807 22:51:26.797905 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:51:26 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:51:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangeradmin_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangeradmin_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangerlookup_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangerlookup_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
I20260807 22:51:27.196887 9995 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIi3wnX/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/postgres -l logfile start
2026-08-07 22:51:29.833 UTC [21646] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:51:29.833 UTC [21646] LOG: listening on IPv4 address "127.9.194.212", port 39029
2026-08-07 22:51:29.838 UTC [21646] LOG: listening on Unix socket "/tmp/.s.PGSQL.39029"
2026-08-07 22:51:29.861 UTC [21649] LOG: database system was shut down at 2026-08-07 22:51:28 UTC
2026-08-07 22:51:29.865 UTC [21646] LOG: database system is ready to accept connections
I20260807 22:51:30.965718 9995 mini_postgres.cc:96] Postgres bound to 39029
2026-08-07 22:51:30.991 UTC [21657] FATAL: database "slave" does not exist
127.9.194.212:39029 - accepting connections
I20260807 22:51:30.992219 9995 mini_ranger.cc:161] Starting Ranger...
I20260807 22:51:31.023264 9995 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:51:31.543596 9995 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:51:31.543718 9995 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin
2026-08-07 22:51:31,885 [I] DB FLAVOR :POSTGRES
2026-08-07 22:51:31,886 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:51:31,886 [I] Checking connection..
2026-08-07 22:51:31,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:51:32,137 [I] Checking connection passed.
2026-08-07 22:51:32,137 [I] --------- Verifying version history table ---------
2026-08-07 22:51:32,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:51:32,379 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:51:32,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:51:32,633 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:51:32,634 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:51:32,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:51:32.900 UTC [21726] WARNING: there is no transaction in progress
2026-08-07 22:51:32,907 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:51:32,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:51:33,188 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:51:33,189 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:51:33,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:51:33,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:51:33,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-07 22:51:33,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:51:34,221 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:51:34,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:51:34,469 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:51:34,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:51:34,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:51:34,969 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:51:34,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:51:35.983 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:35.999 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.017 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.034 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.054 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.075 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.135 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.143 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.158 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.169 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.180 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.194 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.204 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.211 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.218 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36.226 UTC [21837] WARNING: there is no transaction in progress
2026-08-07 22:51:36,886 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:51:36,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:51:37,140 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:51:37,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:51:37,390 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:51:37,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:51:37,658 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:51:37,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:51:37,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:51:38,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-07 22:51:38,419 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:51:38,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:51:38,687 [I] DB_PATCHES have already been applied
I20260807 22:51:38.693517 9995 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIi3wnX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:51:38.693614 9995 mini_ranger.cc:193] Using host: 127.9.194.212
I20260807 22:51:38.714476 9995 mini_ranger.cc:227] Ranger admin URL: http://127.9.194.212:58575
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:51:39.903368 9995 mini_ranger.cc:160] Time spent starting Ranger: real 8.911s user 0.009s sys 0.051s
Aug 07, 2026 10:51:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:51:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 10:51:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:51:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:52:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:52:01.230419 9995 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangerkms_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/rangerkms_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.194.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.194.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:52:01 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:52:01.437975 9995 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:52:01.464880 9995 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:52:02.414147 9995 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:52:02.414356 9995 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms
I20260807 22:52:02.414503 9995 mini_ranger_kms.cc:227] Using postgres at 127.9.194.212:39029
2026-08-07 22:52:02,926 [I] DB FLAVOR :POSTGRES
2026-08-07 22:52:02,927 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:52:02,927 [I] Checking connection
2026-08-07 22:52:02,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:52:03,179 [I] connection success
2026-08-07 22:52:03,180 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:52:03,180 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:52:03,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:52:03,418 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:52:03,419 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:52:03,419 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:52:03,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:39029/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:52:03,709 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:52:04.182473 9995 mini_ranger_kms.cc:310] Created kms service
I20260807 22:52:04.389331 9995 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:52:04.434587 9995 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:52:04 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:04.820676 9995 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:52:04.822201 9995 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:52:04.822274 9995 mini_ranger_kms.cc:240] Using host: 127.9.194.212
I20260807 22:52:04.853230 9995 mini_ranger_kms.cc:276] Ranger KMS PID: 22026
I20260807 22:52:04.853443 9995 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.194.212:47033
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:52:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 22:52:07.367117 9995 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.929s user 0.117s sys 0.436s
I20260807 22:52:07.367525 9995 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:52:07.367769 9995 mini_ranger_kms.cc:401] 127.9.194.212:47033/kms/v1/keys
Aug 07 22:52:07 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786143127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 22:52:07 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:52:08 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:52:10 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:52:10.510337 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38765
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38765
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:47033/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:52:10.855103 22077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:52:10.855466 22077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:52:10.855646 22077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:52:10.863047 22077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:52:10.863198 22077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:52:10.863309 22077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:52:10.863405 22077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:52:10.863520 22077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:52:10.877458 22077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.194.212:47033/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38765
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:52:10.879058 22077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:52:10.880482 22077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:52:10.892931 22082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:10.893193 22083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:10.893232 22086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:10.893921 22077 server_base.cc:1034] running on GCE node
I20260807 22:52:10.894711 22077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:52:10.896190 22077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:52:10.897423 22077 hybrid_clock.cc:648] HybridClock initialized: now 1786143130897387 us; error 44 us; skew 500 ppm
I20260807 22:52:10.897775 22077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:52:10 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:52:10.908010 22077 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:52:10.909613 22077 webserver.cc:460] Webserver started at http://127.9.194.254:40993/ using document root <none> and password file <none>
I20260807 22:52:10.910303 22077 fs_manager.cc:359] Metadata directory not provided
I20260807 22:52:10.910445 22077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:52:10.910792 22077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:52:10 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:10.962575 22077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "c5e5fef9888d4bd4aa498fadf25bdc72"
format_stamp: "Formatted at 2026-08-07 22:52:10 on dist-test-slave-5k54"
server_key: "0d18d5eb310456013ff2d8fffb18afc8"
server_key_iv: "7a41dc30348932d60de55c1315fa510d"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:10.963507 22077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "c5e5fef9888d4bd4aa498fadf25bdc72"
format_stamp: "Formatted at 2026-08-07 22:52:10 on dist-test-slave-5k54"
server_key: "0d18d5eb310456013ff2d8fffb18afc8"
server_key_iv: "7a41dc30348932d60de55c1315fa510d"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:10.969661 22077 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:52:10.986462 22093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:10.988113 22077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:52:10.988297 22077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "c5e5fef9888d4bd4aa498fadf25bdc72"
format_stamp: "Formatted at 2026-08-07 22:52:10 on dist-test-slave-5k54"
server_key: "0d18d5eb310456013ff2d8fffb18afc8"
server_key_iv: "7a41dc30348932d60de55c1315fa510d"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:10.988540 22077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:52:11.006673 22077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:52:11.010609 22077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:52:11.010972 22077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:52:11.030109 22077 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:38765
I20260807 22:52:11.030146 22145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:38765 every 8 connection(s)
I20260807 22:52:11.031543 22077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:52:11.035475 22146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:52:11.040967 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 22077
I20260807 22:52:11.041184 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:52:11.041900 9995 external_mini_cluster.cc:1428] Setting key 2732ffc11b2e7c2b15d8f2d5d13285e2
I20260807 22:52:11.046885 22146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: Bootstrap starting.
I20260807 22:52:11.050738 22146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: Neither blocks nor log segments found. Creating new log.
I20260807 22:52:11.051919 22146 log.cc:824] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: Log is configured to *not* fsync() on all Append() calls
I20260807 22:52:11.055107 22146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: No bootstrap required, opened a new log
Aug 07 22:52:11 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:52:11.063195 22146 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } }
I20260807 22:52:11.063623 22146 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:52:11.063746 22146 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5fef9888d4bd4aa498fadf25bdc72, State: Initialized, Role: FOLLOWER
I20260807 22:52:11.064404 22146 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } }
I20260807 22:52:11.064675 22146 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:52:11.064841 22146 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:52:11.065037 22146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:52:11.065740 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:11.044032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.9.194.254:38765)
0807 22:52:11.044941 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:11.044957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:11.045005 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:52:11.047321 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:11.047347 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:11.047843 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:11.048267 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:11.049331 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:11.051337 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:11.052866 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:11.053750 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:11.053928 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:11.058053 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:11.058061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:11.058093 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:11.058133 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:11.061305 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:11.062035 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:11.062049 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:11.062055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:11.062192 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:11.062568 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:11.062573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:11.062576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:11.063105 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:11.063451 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:11.063988 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 22:52:11.064362 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":421,"threads_started":1}
I20260807 22:52:11.067646 22146 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } }
I20260807 22:52:11.068166 22146 leader_election.cc:302] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e5fef9888d4bd4aa498fadf25bdc72; no voters:
I20260807 22:52:11.068827 22146 leader_election.cc:288] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:52:11.068920 22151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:52:11.069778 22151 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [term 1 LEADER]: Becoming Leader. State: Replica: c5e5fef9888d4bd4aa498fadf25bdc72, State: Running, Role: LEADER
I20260807 22:52:11.070390 22151 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } }
I20260807 22:52:11.070961 22146 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:52:11.073398 22153 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5e5fef9888d4bd4aa498fadf25bdc72. Latest consensus state: current_term: 1 leader_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } } }
I20260807 22:52:11.074082 22152 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e5fef9888d4bd4aa498fadf25bdc72" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38765 } } }
I20260807 22:52:11.074388 22153 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [sys.catalog]: This master's current role is: LEADER
I20260807 22:52:11.074448 22152 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72 [sys.catalog]: This master's current role is: LEADER
I20260807 22:52:11.078579 22161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:52:11.082880 22161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:52:11.083801 22146 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:52:11.107235 22161 catalog_manager.cc:1347] Generated new cluster ID: 76c3933e86314e0c997db95c2ef14457
I20260807 22:52:11.107401 22161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:52:11.123872 22161 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:52:11.125967 22161 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:52:11.141268 22161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: Generated new TSK 0
I20260807 22:52:11.142586 22161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:52:11.928512 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:47033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:52:12 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:52:12.318085 22195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:52:12.318404 22195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:52:12.318578 22195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:52:12.326409 22195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:52:12.326563 22195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:52:12.326769 22195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:52:12.340709 22195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.194.212:47033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:52:12.342514 22195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:52:12.344311 22195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:52:12.362637 22206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:12.363543 22207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:12.369082 22195 server_base.cc:1034] running on GCE node
W20260807 22:52:12.369478 22209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:12.370280 22195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:52:12.371668 22195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:52:12.372948 22195 hybrid_clock.cc:648] HybridClock initialized: now 1786143132372844 us; error 115 us; skew 500 ppm
I20260807 22:52:12.373330 22195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:52:12 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:52:12.377625 22195 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:52:12.379726 22195 webserver.cc:460] Webserver started at http://127.9.194.193:43851/ using document root <none> and password file <none>
I20260807 22:52:12.380640 22195 fs_manager.cc:359] Metadata directory not provided
I20260807 22:52:12.380784 22195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:52:12.381347 22195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:52:12 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:12.398984 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "eef822572b9e45ec9cd02de6c6b69896"
format_stamp: "Formatted at 2026-08-07 22:52:12 on dist-test-slave-5k54"
server_key: "c9d5ca698391e6541a3f2f9b84610abc"
server_key_iv: "c2158712e66c5a94bc55d959042ba94c"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:12.400120 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "eef822572b9e45ec9cd02de6c6b69896"
format_stamp: "Formatted at 2026-08-07 22:52:12 on dist-test-slave-5k54"
server_key: "c9d5ca698391e6541a3f2f9b84610abc"
server_key_iv: "c2158712e66c5a94bc55d959042ba94c"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:12.417965 22195 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.008s sys 0.008s
I20260807 22:52:12.442662 22217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:12.444398 22195 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260807 22:52:12.444607 22195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "eef822572b9e45ec9cd02de6c6b69896"
format_stamp: "Formatted at 2026-08-07 22:52:12 on dist-test-slave-5k54"
server_key: "c9d5ca698391e6541a3f2f9b84610abc"
server_key_iv: "c2158712e66c5a94bc55d959042ba94c"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:12.444869 22195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:52:12.480674 22195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:52:12.486060 22195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:52:12.486474 22195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:52:12.487610 22195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:52:12.495667 22195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:52:12.495771 22195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:12.495915 22195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:52:12.496001 22195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:12.551077 22334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:46785 every 8 connection(s)
I20260807 22:52:12.551131 22195 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:46785
I20260807 22:52:12.553356 22195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:52:12.555756 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 22195
I20260807 22:52:12.555994 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:52:12.556968 9995 external_mini_cluster.cc:1428] Setting key e3ffe043a9bbcc7e301505b1ae4b2096
Aug 07 22:52:12 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:52:12.605396 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:12.560939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:34331 (local address 127.9.194.254:38765)
0807 22:52:12.584245 (+ 23306us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:12.584252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:12.584278 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:52:12.584351 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:12.584357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:12.584467 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:12.584598 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:12.588148 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:12.589373 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:12.591985 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:12.592661 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:12.592764 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:12.597547 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:12.597554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:12.597559 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:12.597604 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:12.601363 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:12.602363 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:12.602369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:12.602373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:12.602541 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:12.603103 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:12.603108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:12.603111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:12.603520 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:12.603656 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:12.604754 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0807 22:52:12.605016 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23158,"thread_start_us":23042,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
I20260807 22:52:12.607798 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:12.557174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38765 (local address 127.9.194.193:34331)
0807 22:52:12.580568 (+ 23394us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:12.580642 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:12.583458 (+ 2816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:12.584971 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:12.584985 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:12.585694 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:52:12.587886 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:12.587914 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:12.589621 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:12.589626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:12.591799 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:12.591815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:12.593201 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:12.593212 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:12.593518 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:12.594394 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:52:12.594427 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:52:12.597271 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:52:12.601572 (+ 4301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:12.601583 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:12.601602 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:12.602221 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:12.602698 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:12.602703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:12.602706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:12.602958 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:12.603764 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:52:12.603772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:52:12.604260 (+ 488us) client_negotiation.cc:769] Sending connection context
0807 22:52:12.605766 (+ 1506us) client_negotiation.cc:240] Negotiation successful
0807 22:52:12.606327 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22994,"thread_start_us":22743,"threads_started":1}
I20260807 22:52:12.610112 22336 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38765
I20260807 22:52:12.610412 22336 heartbeater.cc:464] Registering TS with master...
I20260807 22:52:12.611686 22336 heartbeater.cc:511] Master 127.9.194.254:38765 requested a full tablet report, sending...
I20260807 22:52:12.614356 22110 ts_manager.cc:194] Registered new tserver with Master: eef822572b9e45ec9cd02de6c6b69896 (127.9.194.193:46785)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
I20260807 22:52:12.616588 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:34331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:52:12.659521 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:47033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:52:12 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143132, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
W20260807 22:52:13.157367 22342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:52:13.158084 22342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:52:13.158321 22342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:52:13.170037 22342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:52:13.170271 22342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:52:13.170523 22342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:52:13.186959 22342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.194.212:47033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:52:13.189005 22342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:52:13.191004 22342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:52:13.217875 22349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:13.231968 22350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:13.241134 22342 server_base.cc:1034] running on GCE node
W20260807 22:52:13.241966 22353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:13.242843 22342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:52:13.253657 22342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:52:13.255000 22342 hybrid_clock.cc:648] HybridClock initialized: now 1786143133254931 us; error 73 us; skew 500 ppm
I20260807 22:52:13.255465 22342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:52:13.262705 22342 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:52:13.280583 22342 webserver.cc:460] Webserver started at http://127.9.194.194:39803/ using document root <none> and password file <none>
I20260807 22:52:13.281667 22342 fs_manager.cc:359] Metadata directory not provided
I20260807 22:52:13.281894 22342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:52:13.282330 22342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:13.313609 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "b1a1008b8d1d45ea9876382f1026f905"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "24858eed84ee27fd84021408646330ea"
server_key_iv: "1c184f90aadee74b3252c66ba11f45ca"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.314821 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "b1a1008b8d1d45ea9876382f1026f905"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "24858eed84ee27fd84021408646330ea"
server_key_iv: "1c184f90aadee74b3252c66ba11f45ca"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.327874 22342 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.007s
I20260807 22:52:13.368294 22360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:13.370396 22342 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.004s sys 0.011s
I20260807 22:52:13.370639 22342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "b1a1008b8d1d45ea9876382f1026f905"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "24858eed84ee27fd84021408646330ea"
server_key_iv: "1c184f90aadee74b3252c66ba11f45ca"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.370923 22342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:52:13.420390 22342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:52:13.424801 22342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:52:13.425148 22342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:52:13.426355 22342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:52:13.427915 22342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:52:13.428021 22342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:13.428170 22342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:52:13.428244 22342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:13.482384 22342 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:37131
I20260807 22:52:13.482405 22483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:37131 every 8 connection(s)
I20260807 22:52:13.483804 22342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:52:13.490257 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 22342
I20260807 22:52:13.490446 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:52:13.491026 9995 external_mini_cluster.cc:1428] Setting key 0eafa4c7aec40dd7ae283e224e491ac0
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:52:13.505697 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:13.486576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:34933 (local address 127.9.194.254:38765)
0807 22:52:13.487116 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:13.487123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:13.488354 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 22:52:13.490635 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:13.490641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:13.490738 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:13.490883 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:13.492806 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:13.493850 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:13.495504 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:13.495923 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:13.496054 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:13.499516 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:13.499521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:13.499525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:13.499556 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:13.502295 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:13.503003 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:13.503010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:13.503017 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:13.503152 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:13.503694 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:13.503698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:13.503703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:13.503986 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:13.504147 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:13.505043 (+ 896us) server_negotiation.cc:299] Negotiation successful
0807 22:52:13.505295 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
I20260807 22:52:13.506981 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:13.487262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38765 (local address 127.9.194.194:34933)
0807 22:52:13.488171 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:13.488219 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:13.490326 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:13.491174 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:13.491188 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:13.491909 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:52:13.492628 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:13.492646 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:13.494077 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:13.494081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:13.495332 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:13.495343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:13.496164 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:13.496172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:13.496399 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:13.497104 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:52:13.497125 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:52:13.499246 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:52:13.502435 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:13.502443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:13.502460 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:13.502872 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:13.503267 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:13.503283 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:13.503287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:13.503576 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:13.504116 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:52:13.504124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:52:13.504517 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 22:52:13.504870 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 22:52:13.505465 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":420,"threads_started":1}
I20260807 22:52:13.508508 22484 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38765
I20260807 22:52:13.508859 22484 heartbeater.cc:464] Registering TS with master...
I20260807 22:52:13.509734 22484 heartbeater.cc:511] Master 127.9.194.254:38765 requested a full tablet report, sending...
I20260807 22:52:13.511531 22110 ts_manager.cc:194] Registered new tserver with Master: b1a1008b8d1d45ea9876382f1026f905 (127.9.194.194:37131)
I20260807 22:52:13.512796 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:34933
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:52:13.559002 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:47033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:52:13.627007 22336 heartbeater.cc:503] Master 127.9.194.254:38765 was elected leader, sending a full tablet report...
W20260807 22:52:13.902272 22491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:52:13.902599 22491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:52:13.902761 22491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:52:13.910285 22491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:52:13.910442 22491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:52:13.910607 22491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:52:13.920697 22491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.194.212:47033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:52:13.922016 22491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:52:13.923333 22491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:52:13.937002 22496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:13.937023 22499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:52:13.937077 22497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:52:13.938652 22491 server_base.cc:1034] running on GCE node
I20260807 22:52:13.939304 22491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:52:13.940292 22491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:52:13.941517 22491 hybrid_clock.cc:648] HybridClock initialized: now 1786143133941469 us; error 46 us; skew 500 ppm
I20260807 22:52:13.941823 22491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:52:13.945134 22491 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:52:13.946691 22491 webserver.cc:460] Webserver started at http://127.9.194.195:33605/ using document root <none> and password file <none>
I20260807 22:52:13.947371 22491 fs_manager.cc:359] Metadata directory not provided
I20260807 22:52:13.947553 22491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:52:13.947867 22491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:52:13 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:13.961998 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "44f9dbc1df754f7cad7d6f4094c255a2"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "affc71be270770b828721ea764d5c55d"
server_key_iv: "4ba9c561c6d591b643390bc836f1dd98"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.962888 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "44f9dbc1df754f7cad7d6f4094c255a2"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "affc71be270770b828721ea764d5c55d"
server_key_iv: "4ba9c561c6d591b643390bc836f1dd98"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.969022 22491 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:52:13.986241 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:13.987912 22491 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:52:13.988106 22491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "44f9dbc1df754f7cad7d6f4094c255a2"
format_stamp: "Formatted at 2026-08-07 22:52:13 on dist-test-slave-5k54"
server_key: "affc71be270770b828721ea764d5c55d"
server_key_iv: "4ba9c561c6d591b643390bc836f1dd98"
server_key_version: "kuduclusterkey@0"
I20260807 22:52:13.988353 22491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:52:14.016162 22491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:52:14.020231 22491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:52:14.020560 22491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:52:14.021543 22491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:52:14.022917 22491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:52:14.023020 22491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:14.023169 22491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:52:14.023242 22491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:52:14.056495 22491 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42883
I20260807 22:52:14.056540 22626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42883 every 8 connection(s)
I20260807 22:52:14.057826 22491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:52:14.058202 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 22491
I20260807 22:52:14.058420 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:52:14.059053 9995 external_mini_cluster.cc:1428] Setting key 85d65b940d2d5a920258348d4effef77
Aug 07 22:52:14 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143133, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:52:14.078801 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.060761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:59487 (local address 127.9.194.254:38765)
0807 22:52:14.061337 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.061345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.062581 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 22:52:14.064576 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.064580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.064662 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.064754 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:14.066184 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.066957 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.068520 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.068989 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.069090 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.072527 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:14.072532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:14.072535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:14.072566 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:14.075922 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.076722 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.076726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.076728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.076885 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.077355 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.077358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.077360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.077669 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:14.077844 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.078385 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.078542 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":295,"threads_started":1}
I20260807 22:52:14.079859 22631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.061480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38765 (local address 127.9.194.195:59487)
0807 22:52:14.062392 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.062446 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.064327 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.064961 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.064969 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.065392 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:52:14.066022 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.066041 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.067100 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.067104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.068383 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.068394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.069175 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.069180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.069365 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.070037 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:52:14.070059 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:52:14.072330 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:52:14.076082 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:14.076089 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:14.076101 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:14.076596 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:14.076975 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:52:14.076978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:52:14.076980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:52:14.077253 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:52:14.077775 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:52:14.077781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:52:14.078078 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.078292 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.078652 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":342,"threads_started":1}
I20260807 22:52:14.081149 22627 heartbeater.cc:347] Connected to a master server at 127.9.194.254:38765
I20260807 22:52:14.081439 22627 heartbeater.cc:464] Registering TS with master...
I20260807 22:52:14.082149 22627 heartbeater.cc:511] Master 127.9.194.254:38765 requested a full tablet report, sending...
I20260807 22:52:14.083547 22110 ts_manager.cc:194] Registered new tserver with Master: 44f9dbc1df754f7cad7d6f4094c255a2 (127.9.194.195:42883)
I20260807 22:52:14.084312 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:59487
I20260807 22:52:14.087376 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:52:14.104423 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.092075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.9.194.254:38765)
0807 22:52:14.092294 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.092300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.092716 (+ 416us) server_negotiation.cc:407] Connection header received
0807 22:52:14.092853 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.092856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.092926 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.093015 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:14.094601 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.095496 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.096779 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.097199 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.097286 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.099515 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:14.099520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:14.099523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:14.099553 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:14.102095 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.102692 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.102696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.102698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.102815 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.103149 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.103153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.103154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.103397 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:14.103644 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.103902 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.104083 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:52:14.108353 22110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:52:14.115716 22110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:52:14.151638 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.129591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56378 (local address 127.9.194.195:42883)
0807 22:52:14.143070 (+ 13479us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.143082 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.143123 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:52:14.143246 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.143256 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.143581 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.143805 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:52:14.144413 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.145671 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.148790 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.150422 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.150555 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.150787 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.151073 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.151245 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13272,"thread_start_us":13110,"threads_started":1}
I20260807 22:52:14.151682 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.128942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42883 (local address 127.0.0.1:56378)
0807 22:52:14.141381 (+ 12439us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.141441 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.141637 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.143832 (+ 2195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.143837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.143862 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:52:14.144217 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.144226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.145854 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.145861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.148590 (+ 2729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.148604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.150641 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.150647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.150834 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.150880 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.151022 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.151221 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12116,"mutex_wait_us":1,"thread_start_us":12907,"threads_started":1}
I20260807 22:52:14.153659 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.133544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:46785 (local address 127.0.0.1:36978)
0807 22:52:14.145210 (+ 11666us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.145227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.145362 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.146256 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.146265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.146291 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:52:14.147042 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.147056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.148992 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.149000 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.151215 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.151227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.152806 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.152812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.152965 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.152995 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.153091 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.153240 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11538,"mutex_wait_us":104,"thread_start_us":11408,"threads_started":1}
I20260807 22:52:14.153659 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.134382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.9.194.193:46785)
0807 22:52:14.144738 (+ 10356us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.144749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.145359 (+ 610us) server_negotiation.cc:407] Connection header received
0807 22:52:14.145498 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.145506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.145765 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.145979 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:52:14.147290 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.148861 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.151392 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.152584 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.152730 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.152959 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.153148 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.153310 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10149,"thread_start_us":9963,"threads_started":1}
I20260807 22:52:14.154093 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.129192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:37131 (local address 127.0.0.1:37228)
0807 22:52:14.143160 (+ 13968us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.143175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.143266 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.146473 (+ 3207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.146478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.146504 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:52:14.147187 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.147198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.148562 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.148579 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.150968 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.151003 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.153252 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.153258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.153440 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.153470 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.153569 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.153712 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13872,"mutex_wait_us":1,"thread_start_us":14123,"threads_started":1}
I20260807 22:52:14.154681 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.129502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.9.194.194:37131)
0807 22:52:14.145867 (+ 16365us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.145875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.145902 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:52:14.145995 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.146002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.146252 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.146485 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:52:14.147362 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.148411 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.151185 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.152693 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.152907 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.153115 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.154297 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.154433 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16175,"thread_start_us":16583,"threads_started":1}
I20260807 22:52:14.156484 22561 tablet_service.cc:1467] Processing CreateTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 (DEFAULT_TABLE table=test-table [id=c4cd161836124286a4f5a15c53ab7dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:52:14.157238 22268 tablet_service.cc:1467] Processing CreateTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 (DEFAULT_TABLE table=test-table [id=c4cd161836124286a4f5a15c53ab7dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:52:14.158095 22561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97de5c5b20e4e6c8d063471a30907e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:52:14.158481 22268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97de5c5b20e4e6c8d063471a30907e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:52:14.158360 22418 tablet_service.cc:1467] Processing CreateTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 (DEFAULT_TABLE table=test-table [id=c4cd161836124286a4f5a15c53ab7dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:52:14.159641 22418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97de5c5b20e4e6c8d063471a30907e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:52:14.169615 22647 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Bootstrap starting.
I20260807 22:52:14.173288 22648 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Bootstrap starting.
I20260807 22:52:14.173390 22647 tablet_bootstrap.cc:654] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Neither blocks nor log segments found. Creating new log.
I20260807 22:52:14.174711 22647 log.cc:824] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Log is configured to *not* fsync() on all Append() calls
I20260807 22:52:14.175338 22649 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Bootstrap starting.
I20260807 22:52:14.177644 22648 tablet_bootstrap.cc:654] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Neither blocks nor log segments found. Creating new log.
I20260807 22:52:14.178534 22649 tablet_bootstrap.cc:654] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Neither blocks nor log segments found. Creating new log.
I20260807 22:52:14.179201 22648 log.cc:824] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:52:14.179628 22649 log.cc:824] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Log is configured to *not* fsync() on all Append() calls
I20260807 22:52:14.182231 22647 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: No bootstrap required, opened a new log
I20260807 22:52:14.182516 22647 ts_tablet_manager.cc:1397] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260807 22:52:14.182629 22649 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: No bootstrap required, opened a new log
I20260807 22:52:14.182812 22649 ts_tablet_manager.cc:1397] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 22:52:14.185611 22648 tablet_bootstrap.cc:492] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: No bootstrap required, opened a new log
I20260807 22:52:14.185881 22648 ts_tablet_manager.cc:1397] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260807 22:52:14.190002 22649 raft_consensus.cc:348] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.190393 22649 raft_consensus.cc:374] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:52:14.190498 22649 raft_consensus.cc:729] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a1008b8d1d45ea9876382f1026f905, State: Initialized, Role: FOLLOWER
I20260807 22:52:14.191104 22649 consensus_queue.cc:260] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.192890 22647 raft_consensus.cc:348] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.193197 22649 ts_tablet_manager.cc:1428] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Time spent starting tablet: real 0.010s user 0.010s sys 0.002s
I20260807 22:52:14.193423 22647 raft_consensus.cc:374] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:52:14.193573 22647 raft_consensus.cc:729] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef822572b9e45ec9cd02de6c6b69896, State: Initialized, Role: FOLLOWER
I20260807 22:52:14.193598 22484 heartbeater.cc:503] Master 127.9.194.254:38765 was elected leader, sending a full tablet report...
I20260807 22:52:14.194160 22647 consensus_queue.cc:260] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.195972 22647 ts_tablet_manager.cc:1428] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 22:52:14.196401 22648 raft_consensus.cc:348] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.196801 22648 raft_consensus.cc:374] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:52:14.196941 22648 raft_consensus.cc:729] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f9dbc1df754f7cad7d6f4094c255a2, State: Initialized, Role: FOLLOWER
I20260807 22:52:14.197523 22648 consensus_queue.cc:260] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.198555 22627 heartbeater.cc:503] Master 127.9.194.254:38765 was elected leader, sending a full tablet report...
I20260807 22:52:14.199266 22648 ts_tablet_manager.cc:1428] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260807 22:52:14.219615 22654 raft_consensus.cc:482] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:52:14.219877 22654 raft_consensus.cc:504] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.221537 22654 leader_election.cc:288] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44f9dbc1df754f7cad7d6f4094c255a2 (127.9.194.195:42883), b1a1008b8d1d45ea9876382f1026f905 (127.9.194.194:37131)
I20260807 22:52:14.228789 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.222186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:43157 (local address 127.9.194.194:37131)
0807 22:52:14.222428 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.222436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.223173 (+ 737us) server_negotiation.cc:407] Connection header received
0807 22:52:14.223346 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.223349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.223407 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.223541 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:52:14.224234 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.224962 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.226838 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.227746 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.227866 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.227919 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.228378 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.228521 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:52:14.228858 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.222066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:42799 (local address 127.9.194.195:42883)
0807 22:52:14.222302 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.222310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.223067 (+ 757us) server_negotiation.cc:407] Connection header received
0807 22:52:14.223303 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.223309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.223380 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.223530 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:52:14.224157 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.224928 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.226886 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.227912 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.228079 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.228156 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.228473 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.228623 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:52:14.228883 22657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.222033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:37131 (local address 127.9.194.193:43157)
0807 22:52:14.223069 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.223083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.223185 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.223552 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.223557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.223645 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:52:14.224041 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.224051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.225118 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.225125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.226678 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.226695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.227973 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.227983 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.228177 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.228218 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.228295 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.228412 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":19,"spinlock_wait_cycles":21760,"thread_start_us":632,"threads_started":1}
I20260807 22:52:14.229127 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.221955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42883 (local address 127.9.194.193:42799)
0807 22:52:14.222955 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0807 22:52:14.222970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:52:14.223135 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:52:14.223547 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:52:14.223551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:52:14.223575 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:52:14.224001 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.224014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.225062 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.225070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.226698 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:52:14.226710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.228118 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.228123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:52:14.228290 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.228329 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:52:14.228434 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:52:14.228561 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":86,"thread_start_us":671,"threads_started":1}
I20260807 22:52:14.230094 22438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97de5c5b20e4e6c8d063471a30907e4" candidate_uuid: "eef822572b9e45ec9cd02de6c6b69896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a1008b8d1d45ea9876382f1026f905" is_pre_election: true
I20260807 22:52:14.230094 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97de5c5b20e4e6c8d063471a30907e4" candidate_uuid: "eef822572b9e45ec9cd02de6c6b69896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" is_pre_election: true
I20260807 22:52:14.230670 22581 raft_consensus.cc:2454] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eef822572b9e45ec9cd02de6c6b69896 in term 0.
I20260807 22:52:14.230670 22438 raft_consensus.cc:2454] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eef822572b9e45ec9cd02de6c6b69896 in term 0.
I20260807 22:52:14.231357 22221 leader_election.cc:302] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1a1008b8d1d45ea9876382f1026f905, eef822572b9e45ec9cd02de6c6b69896; no voters:
I20260807 22:52:14.231778 22654 raft_consensus.cc:2781] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:52:14.231983 22654 raft_consensus.cc:482] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:52:14.232106 22654 raft_consensus.cc:3037] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:52:14.234301 22654 raft_consensus.cc:504] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.234921 22654 leader_election.cc:288] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [CANDIDATE]: Term 1 election: Requested vote from peers 44f9dbc1df754f7cad7d6f4094c255a2 (127.9.194.195:42883), b1a1008b8d1d45ea9876382f1026f905 (127.9.194.194:37131)
I20260807 22:52:14.235351 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97de5c5b20e4e6c8d063471a30907e4" candidate_uuid: "eef822572b9e45ec9cd02de6c6b69896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9dbc1df754f7cad7d6f4094c255a2"
I20260807 22:52:14.235477 22438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97de5c5b20e4e6c8d063471a30907e4" candidate_uuid: "eef822572b9e45ec9cd02de6c6b69896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a1008b8d1d45ea9876382f1026f905"
I20260807 22:52:14.235601 22581 raft_consensus.cc:3037] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:52:14.235741 22438 raft_consensus.cc:3037] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:52:14.236093 22485 tablet.cc:2369] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:52:14.238201 22438 raft_consensus.cc:2454] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef822572b9e45ec9cd02de6c6b69896 in term 1.
I20260807 22:52:14.238220 22581 raft_consensus.cc:2454] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef822572b9e45ec9cd02de6c6b69896 in term 1.
I20260807 22:52:14.238641 22219 leader_election.cc:302] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44f9dbc1df754f7cad7d6f4094c255a2, b1a1008b8d1d45ea9876382f1026f905, eef822572b9e45ec9cd02de6c6b69896; no voters:
I20260807 22:52:14.238986 22654 raft_consensus.cc:2781] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:52:14.239501 22654 raft_consensus.cc:686] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 1 LEADER]: Becoming Leader. State: Replica: eef822572b9e45ec9cd02de6c6b69896, State: Running, Role: LEADER
I20260807 22:52:14.240018 22654 consensus_queue.cc:237] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } }
I20260807 22:52:14.245019 22110 catalog_manager.cc:5495] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 reported cstate change: term changed from 0 to 1, leader changed from <none> to eef822572b9e45ec9cd02de6c6b69896 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "eef822572b9e45ec9cd02de6c6b69896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eef822572b9e45ec9cd02de6c6b69896" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 46785 } health_report { overall_health: HEALTHY } } }
I20260807 22:52:14.277926 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.272234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.9.194.193:46785)
0807 22:52:14.272424 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.272429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.272447 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:52:14.272638 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.272641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.272717 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.272840 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:52:14.273255 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.273985 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.275732 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.276212 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.276302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.276817 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:52:14.277407 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:52:14.277558 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.277632 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.277716 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:52:14.285892 22581 raft_consensus.cc:1261] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 1 FOLLOWER]: Refusing update from remote peer eef822572b9e45ec9cd02de6c6b69896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:52:14.285888 22438 raft_consensus.cc:1261] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 1 FOLLOWER]: Refusing update from remote peer eef822572b9e45ec9cd02de6c6b69896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:52:14.286827 22658 consensus_queue.cc:1035] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44f9dbc1df754f7cad7d6f4094c255a2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:52:14.287237 22654 consensus_queue.cc:1035] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a1008b8d1d45ea9876382f1026f905" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:52:14.300977 22662 mvcc.cc:205] Tried to move back new op lower bound from 7316042278025461760 to 7316042277851779072. Current Snapshot: MvccSnapshot[applied={T|T < 7316042278025461760}]
I20260807 22:52:14.314109 22109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:52:14.314427 22109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:52:14.318161 22109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5e5fef9888d4bd4aa498fadf25bdc72: Sending DeleteTablet for 3 replicas of tablet a97de5c5b20e4e6c8d063471a30907e4
I20260807 22:52:14.319283 22561 tablet_service.cc:1514] Processing DeleteTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:52:14 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:56378
I20260807 22:52:14.319288 22418 tablet_service.cc:1514] Processing DeleteTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:52:14 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37228
I20260807 22:52:14.319625 22268 tablet_service.cc:1514] Processing DeleteTablet for tablet a97de5c5b20e4e6c8d063471a30907e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:52:14 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:36978
I20260807 22:52:14.322263 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 22195
I20260807 22:52:14.332293 22674 tablet_replica.cc:330] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: stopping tablet replica
I20260807 22:52:14.332887 22674 raft_consensus.cc:2229] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:52:14.333766 22674 raft_consensus.cc:2258] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:52:14.334669 22673 tablet_replica.cc:330] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: stopping tablet replica
I20260807 22:52:14.335124 22673 raft_consensus.cc:2229] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:52:14.335762 22673 raft_consensus.cc:2258] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:52:14.336717 22672 tablet_replica.cc:330] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: stopping tablet replica
I20260807 22:52:14.337189 22672 raft_consensus.cc:2229] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:52:14.337698 22672 raft_consensus.cc:2258] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:52:14.337769 22674 ts_tablet_manager.cc:1905] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:52:14.337957 22673 ts_tablet_manager.cc:1905] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:52:14.341509 22672 ts_tablet_manager.cc:1905] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:52:14.345100 22673 ts_tablet_manager.cc:1918] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:52:14.345332 22673 log.cc:1196] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/a97de5c5b20e4e6c8d063471a30907e4
I20260807 22:52:14.345949 22673 ts_tablet_manager.cc:1939] T a97de5c5b20e4e6c8d063471a30907e4 P b1a1008b8d1d45ea9876382f1026f905: Deleting consensus metadata
Aug 07 22:52:14 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:52:14.347961 22097 catalog_manager.cc:4841] TS b1a1008b8d1d45ea9876382f1026f905 (127.9.194.194:37131): tablet a97de5c5b20e4e6c8d063471a30907e4 (table test-table [id=c4cd161836124286a4f5a15c53ab7dfc]) successfully deleted
I20260807 22:52:14.348656 22672 ts_tablet_manager.cc:1918] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:52:14.348855 22672 log.cc:1196] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/a97de5c5b20e4e6c8d063471a30907e4
I20260807 22:52:14.349462 22672 ts_tablet_manager.cc:1939] T a97de5c5b20e4e6c8d063471a30907e4 P 44f9dbc1df754f7cad7d6f4094c255a2: Deleting consensus metadata
I20260807 22:52:14.351516 22095 catalog_manager.cc:4841] TS 44f9dbc1df754f7cad7d6f4094c255a2 (127.9.194.195:42883): tablet a97de5c5b20e4e6c8d063471a30907e4 (table test-table [id=c4cd161836124286a4f5a15c53ab7dfc]) successfully deleted
I20260807 22:52:14.352073 22674 ts_tablet_manager.cc:1918] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:52:14.352247 22674 log.cc:1196] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/a97de5c5b20e4e6c8d063471a30907e4
I20260807 22:52:14.352891 22674 ts_tablet_manager.cc:1939] T a97de5c5b20e4e6c8d063471a30907e4 P eef822572b9e45ec9cd02de6c6b69896: Deleting consensus metadata
I20260807 22:52:14.354764 22094 catalog_manager.cc:4841] TS eef822572b9e45ec9cd02de6c6b69896 (127.9.194.193:46785): tablet a97de5c5b20e4e6c8d063471a30907e4 (table test-table [id=c4cd161836124286a4f5a15c53ab7dfc]) successfully deleted
I20260807 22:52:14.357182 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.323852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.9.194.193:46785)
0807 22:52:14.324017 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.324024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.328375 (+ 4351us) server_negotiation.cc:407] Connection header received
0807 22:52:14.328598 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.328603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.328693 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.328814 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:14.336254 (+ 7440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.337404 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.339881 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.340547 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.340677 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.349893 (+ 9216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:14.349909 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:14.349926 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:14.349972 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:14.353986 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.354884 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.354893 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.354897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.355068 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.355503 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.355509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.355512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.355969 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:14.356293 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.356540 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.356796 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:52:14.357859 22330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:52:14.459159 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 22195
I20260807 22:52:14.480149 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 22342
Aug 07 22:52:14 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:52:14.498943 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.480961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.9.194.194:37131)
0807 22:52:14.481150 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.481155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.481173 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:52:14.481357 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.481364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.481442 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.481552 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:14.483072 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.484268 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.485597 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.486236 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.486360 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.491739 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:14.491752 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:14.491768 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:14.491807 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:14.495998 (+ 4191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.496768 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.496777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.496781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.496937 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.497287 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.497292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.497296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.497735 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:14.498043 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.498310 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.498549 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:52:14.499825 22479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:52:14.592774 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 22342
I20260807 22:52:14.609943 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 22491
Aug 07 22:52:14 dist-test-slave-5k54 krb5kdc[21615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:52:14.624871 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:52:14.610765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.9.194.195:42883)
0807 22:52:14.610956 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 22:52:14.610961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:52:14.610979 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:52:14.611181 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:52:14.611185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:52:14.611248 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:52:14.611341 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:52:14.612750 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.613459 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.614820 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:52:14.615199 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:52:14.615290 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:52:14.619746 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:52:14.619760 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:52:14.619777 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:52:14.619809 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:52:14.622350 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.622962 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.622969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.622971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.623082 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:52:14.623403 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:52:14.623406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:52:14.623408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:52:14.623837 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:52:14.624053 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 22:52:14.624311 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 22:52:14.624469 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:52:14.625545 22622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:52:14.716399 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 22491
I20260807 22:52:14.735277 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 22077
I20260807 22:52:14.736642 22141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:52:14.850881 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 22077
I20260807 22:52:14.882323 9995 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:52:15.227872 9995 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:52:15.228217 9995 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:52:15.573849 9995 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:52:15.574 UTC [21646] LOG: received smart shutdown request
2026-08-07 22:52:15.578 UTC [21646] LOG: background worker "logical replication launcher" (PID 21655) exited with exit code 1
2026-08-07 22:52:15.579 UTC [21650] LOG: shutting down
2026-08-07 22:52:15.757 UTC [21646] LOG: database system is shut down
2026-08-07T22:52:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50296 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:52:16 dist-test-slave-5k54 krb5kdc[22689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:52:16 dist-test-slave-5k54 krb5kdc[22689](info): set up 2 sockets
Aug 07 22:52:16 dist-test-slave-5k54 krb5kdc[22689](info): commencing operation
krb5kdc: starting...
W20260807 22:52:17.076947 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:52:17 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:52:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangeradmin_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangeradmin_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangerlookup_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangerlookup_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
I20260807 22:52:17.443269 9995 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIi3wnX/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/postgres -l logfile start
2026-08-07 22:52:19.932 UTC [22720] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:52:19.932 UTC [22720] LOG: listening on IPv4 address "127.9.194.212", port 42975
2026-08-07 22:52:19.935 UTC [22720] LOG: listening on Unix socket "/tmp/.s.PGSQL.42975"
2026-08-07 22:52:19.958 UTC [22723] LOG: database system was shut down at 2026-08-07 22:52:18 UTC
2026-08-07 22:52:19.961 UTC [22720] LOG: database system is ready to accept connections
I20260807 22:52:20.976374 9995 mini_postgres.cc:96] Postgres bound to 42975
2026-08-07 22:52:20.997 UTC [22731] FATAL: database "slave" does not exist
127.9.194.212:42975 - accepting connections
I20260807 22:52:20.998514 9995 mini_ranger.cc:161] Starting Ranger...
I20260807 22:52:21.027624 9995 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:52:21.487491 9995 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:52:21.487623 9995 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin
2026-08-07 22:52:21,798 [I] DB FLAVOR :POSTGRES
2026-08-07 22:52:21,799 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:52:21,799 [I] Checking connection..
2026-08-07 22:52:21,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:52:22,065 [I] Checking connection passed.
2026-08-07 22:52:22,066 [I] --------- Verifying version history table ---------
2026-08-07 22:52:22,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:52:22,311 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:52:22,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:52:22,562 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:52:22,562 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:52:22,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:52:22.825 UTC [22799] WARNING: there is no transaction in progress
2026-08-07 22:52:22,832 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:52:22,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:52:23,073 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:52:23,073 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:52:23,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:52:23,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:52:23,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-07 22:52:23,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:52:24,121 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:52:24,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:52:24,392 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:52:24,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:52:24,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:52:24,912 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:52:24,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:52:25.875 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:25.892 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:25.908 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:25.924 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:25.940 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:25.958 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.011 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.018 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.031 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.041 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.051 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.063 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.072 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.078 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.085 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26.092 UTC [22913] WARNING: there is no transaction in progress
2026-08-07 22:52:26,719 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:52:26,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:52:26,969 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:52:26,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:52:27,246 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:52:27,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:52:27,497 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:52:27,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:52:27,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:52:28,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-07 22:52:28,262 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:52:28,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:52:28,530 [I] DB_PATCHES have already been applied
I20260807 22:52:28.536921 9995 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIi3wnX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:52:28.537011 9995 mini_ranger.cc:193] Using host: 127.9.194.212
I20260807 22:52:28.556272 9995 mini_ranger.cc:227] Ranger admin URL: http://127.9.194.212:33667
Aug 07, 2026 10:52:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:52:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:52:29.748713 9995 mini_ranger.cc:160] Time spent starting Ranger: real 8.750s user 0.008s sys 0.047s
Aug 07, 2026 10:52:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:52:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 10:52:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:52:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:52:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:52:52.002224 9995 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.194.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.194.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangerkms_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.194.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/rangerkms_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.194.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.194.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.194.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/HTTP_127.9.194.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:52:52 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143172, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:52:52.215853 9995 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:52:52.240942 9995 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:52:53.151858 9995 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:52:53.152001 9995 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms
I20260807 22:52:53.152063 9995 mini_ranger_kms.cc:227] Using postgres at 127.9.194.212:42975
2026-08-07 22:52:53,519 [I] DB FLAVOR :POSTGRES
2026-08-07 22:52:53,520 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:52:53,520 [I] Checking connection
2026-08-07 22:52:53,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:52:53,763 [I] connection success
2026-08-07 22:52:53,763 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:52:53,764 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:52:53,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:52:54,022 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:52:54,022 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:52:54,022 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:52:54,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.194.212:42975/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:52:54,274 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:52:54.758311 9995 mini_ranger_kms.cc:310] Created kms service
I20260807 22:52:54.978834 9995 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:52:55.026796 9995 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:52:55 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143172, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:52:55.430519 9995 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:52:55.430850 9995 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIi3wnX/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIi3wnX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:52:55.430893 9995 mini_ranger_kms.cc:240] Using host: 127.9.194.212
I20260807 22:52:55.455132 9995 mini_ranger_kms.cc:276] Ranger KMS PID: 23108
I20260807 22:52:55.455349 9995 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.194.212:42389
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:52:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 22:52:57 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786143177, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:52:57.869719 9995 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.654s user 0.060s sys 0.336s
I20260807 22:52:57.869989 9995 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:52:57.870189 9995 mini_ranger_kms.cc:401] 127.9.194.212:42389/kms/v1/keys
Aug 07 22:52:58 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143177, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:52:58 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143177, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.194.212@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:53:00 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:53:00.781580 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43673
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43673
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:01.119594 23161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:01.119871 23161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:01.120069 23161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:01.127107 23161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:01.127255 23161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:01.127332 23161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:01.127383 23161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:01.127481 23161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:01.137311 23161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.194.212:42389/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43673
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:01.138566 23161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:01.140003 23161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:01.151412 23166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:01.151506 23167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:01.151466 23170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:01.152498 23161 server_base.cc:1034] running on GCE node
I20260807 22:53:01.153226 23161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:01.154536 23161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:01.155753 23161 hybrid_clock.cc:648] HybridClock initialized: now 1786143181155721 us; error 43 us; skew 500 ppm
I20260807 22:53:01.156100 23161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:01 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143181, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:01.159128 23161 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:01.160602 23161 webserver.cc:460] Webserver started at http://127.9.194.254:45955/ using document root <none> and password file <none>
I20260807 22:53:01.161211 23161 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:01.161301 23161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:01.161552 23161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:53:01 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143181, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:01.220418 23161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "d835294dfe084b7e90ca1df886512f1b"
format_stamp: "Formatted at 2026-08-07 22:53:01 on dist-test-slave-5k54"
server_key: "88e37aca7270251a2d3bc5109370b7f7"
server_key_iv: "5335b5aa34eff0468bbf7ab5ee83d9b2"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:01.221660 23161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "d835294dfe084b7e90ca1df886512f1b"
format_stamp: "Formatted at 2026-08-07 22:53:01 on dist-test-slave-5k54"
server_key: "88e37aca7270251a2d3bc5109370b7f7"
server_key_iv: "5335b5aa34eff0468bbf7ab5ee83d9b2"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:01.228888 23161 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260807 22:53:01.250289 23177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:01.252133 23161 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:53:01.252319 23161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d835294dfe084b7e90ca1df886512f1b"
format_stamp: "Formatted at 2026-08-07 22:53:01 on dist-test-slave-5k54"
server_key: "88e37aca7270251a2d3bc5109370b7f7"
server_key_iv: "5335b5aa34eff0468bbf7ab5ee83d9b2"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:01.252579 23161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:01.287371 23161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:01.291895 23161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:01.292240 23161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:01.312230 23161 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43673
I20260807 22:53:01.312289 23235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43673 every 8 connection(s)
I20260807 22:53:01.313659 23161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:01.318341 23236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:01.320798 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23161
I20260807 22:53:01.321050 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:53:01.321732 9995 external_mini_cluster.cc:1428] Setting key a2c950e0585a0f300711ef3ab95a9ddd
I20260807 22:53:01.333094 23236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Bootstrap starting.
I20260807 22:53:01.337167 23236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:01.338608 23236 log.cc:824] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:01.341861 23236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: No bootstrap required, opened a new log
Aug 07 22:53:01 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:01.352366 23236 raft_consensus.cc:348] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:01.352859 23236 raft_consensus.cc:374] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:01.352988 23236 raft_consensus.cc:729] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835294dfe084b7e90ca1df886512f1b, State: Initialized, Role: FOLLOWER
I20260807 22:53:01.353696 23236 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:01.353952 23236 raft_consensus.cc:388] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:01.354143 23236 raft_consensus.cc:482] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:01.354357 23236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:01.357208 23236 raft_consensus.cc:504] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:01.357818 23236 leader_election.cc:302] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d835294dfe084b7e90ca1df886512f1b; no voters:
I20260807 22:53:01.358697 23236 leader_election.cc:288] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:53:01.358883 23241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:01.363278 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:01.324121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.9.194.254:43673)
0807 22:53:01.335986 (+ 11865us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:01.336002 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:01.336460 (+ 458us) server_negotiation.cc:407] Connection header received
0807 22:53:01.338562 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:01.338586 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:01.339034 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:01.339556 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:01.341282 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:01.343543 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:01.345632 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:01.346713 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:01.346903 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:01.351981 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:01.351992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:01.352010 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:01.352056 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:01.356275 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:01.357161 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:01.357170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:01.357178 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:01.357377 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:01.357843 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:01.357848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:01.357852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:01.358766 (+ 914us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:01.360574 (+ 1808us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:01.360973 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:53:01.361528 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11583,"thread_start_us":11264,"threads_started":1}
I20260807 22:53:01.371783 23241 raft_consensus.cc:686] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 LEADER]: Becoming Leader. State: Replica: d835294dfe084b7e90ca1df886512f1b, State: Running, Role: LEADER
I20260807 22:53:01.372462 23241 consensus_queue.cc:237] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:01.375113 23236 sys_catalog.cc:564] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:01.376487 23243 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d835294dfe084b7e90ca1df886512f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } } }
I20260807 22:53:01.376868 23243 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:01.377476 23242 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835294dfe084b7e90ca1df886512f1b. Latest consensus state: current_term: 1 leader_uuid: "d835294dfe084b7e90ca1df886512f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } } }
I20260807 22:53:01.377756 23242 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:01.398617 23246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:01.406337 23246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:01.407065 23236 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:53:01.426231 23246 catalog_manager.cc:1347] Generated new cluster ID: 00bfaaea59054a76a0736161e0371ecc
I20260807 22:53:01.426389 23246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:01.444595 23246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:53:01.446076 23246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:01.459344 23246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Generated new TSK 0
I20260807 22:53:01.460604 23246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:53:02.231769 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:02 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:53:02.636878 23284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:02.637209 23284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:02.637410 23284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:02.645308 23284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:02.645467 23284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:02.645663 23284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:02.656208 23284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:02.657786 23284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:02.659201 23284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:02.677331 23296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:02.684669 23295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:02.684753 23299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:02.686039 23284 server_base.cc:1034] running on GCE node
I20260807 22:53:02.686918 23284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:02.688383 23284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:02.689690 23284 hybrid_clock.cc:648] HybridClock initialized: now 1786143182689620 us; error 69 us; skew 500 ppm
I20260807 22:53:02.690085 23284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:02 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:02.698374 23284 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:02.702919 23284 webserver.cc:460] Webserver started at http://127.9.194.193:37865/ using document root <none> and password file <none>
I20260807 22:53:02.703887 23284 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:02.704041 23284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:02.704402 23284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:53:02 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:02.724255 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "cb6338c7953647a3a08f6929f35f6d63"
format_stamp: "Formatted at 2026-08-07 22:53:02 on dist-test-slave-5k54"
server_key: "95a441baf5f9ff3938df4f04c0fedd08"
server_key_iv: "91e6ebe2a0d04285139c8d3cf0db7443"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:02.725454 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "cb6338c7953647a3a08f6929f35f6d63"
format_stamp: "Formatted at 2026-08-07 22:53:02 on dist-test-slave-5k54"
server_key: "95a441baf5f9ff3938df4f04c0fedd08"
server_key_iv: "91e6ebe2a0d04285139c8d3cf0db7443"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:02.749301 23284 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.011s sys 0.007s
I20260807 22:53:02.778126 23307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:02.781809 23284 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.004s
I20260807 22:53:02.782033 23284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "cb6338c7953647a3a08f6929f35f6d63"
format_stamp: "Formatted at 2026-08-07 22:53:02 on dist-test-slave-5k54"
server_key: "95a441baf5f9ff3938df4f04c0fedd08"
server_key_iv: "91e6ebe2a0d04285139c8d3cf0db7443"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:02.782296 23284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:02.820504 23284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:02.825778 23284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:02.826112 23284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:02.827291 23284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:02.829128 23284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:02.829221 23284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:02.829344 23284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:02.829404 23284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:02.928753 23284 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39607
I20260807 22:53:02.928776 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39607 every 8 connection(s)
I20260807 22:53:02.930294 23284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:02.933286 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23284
I20260807 22:53:02.933498 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:53:02.934336 9995 external_mini_cluster.cc:1428] Setting key bf8e6b90dfd3d51312f5652eead4f722
Aug 07 22:53:02 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:02.964388 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:02.934504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.193:41091)
0807 22:53:02.938795 (+ 4291us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:02.938853 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:02.941251 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:02.944646 (+ 3395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:02.944658 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:02.945330 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:02.946177 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:02.946204 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:02.947787 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:02.947793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:02.950233 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:02.950253 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:02.951614 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:02.951621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:02.951897 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:02.952708 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:02.952743 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:02.955549 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:02.959779 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:02.959788 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:02.959806 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:02.960364 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:02.960798 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:02.960803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:02.960807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:02.961031 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:02.961725 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:02.961734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:02.962147 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 22:53:02.962450 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 22:53:02.962976 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3998,"thread_start_us":3737,"threads_started":1}
I20260807 22:53:02.965715 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:02.935669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:41091 (local address 127.9.194.254:43673)
0807 22:53:02.943876 (+ 8207us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:02.943883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:02.943906 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:53:02.943968 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:02.943973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:02.944058 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:02.944177 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:02.946398 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:02.947600 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:02.950450 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:02.951142 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:02.951274 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:02.955830 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:02.955837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:02.955842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:02.955881 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:02.959536 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:02.960523 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:02.960529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:02.960533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:02.960675 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:02.961159 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:02.961164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:02.961168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:02.961520 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:02.961647 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:02.965117 (+ 3470us) server_negotiation.cc:299] Negotiation successful
0807 22:53:02.965338 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8071,"thread_start_us":7957,"threads_started":1}
I20260807 22:53:02.967368 23426 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:02.967711 23426 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:02.968546 23426 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:02.971169 23196 ts_manager.cc:194] Registered new tserver with Master: cb6338c7953647a3a08f6929f35f6d63 (127.9.194.193:39607)
I20260807 22:53:02.973225 23196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:41091
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:53:03.030284 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:03 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:53:03 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
W20260807 22:53:03.504942 23434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:03.505455 23434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:03.505677 23434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:03.514137 23434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:03.514339 23434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:03.514583 23434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:03.530134 23434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:03.532156 23434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:03.533952 23434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:03.574065 23441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:03.574610 23442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:03.583115 23434 server_base.cc:1034] running on GCE node
W20260807 22:53:03.583896 23445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:03.584892 23434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:03.591750 23434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:03.602532 23434 hybrid_clock.cc:648] HybridClock initialized: now 1786143183598930 us; error 3589 us; skew 500 ppm
I20260807 22:53:03.602989 23434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:03 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143183, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:03.607736 23434 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:03.646498 23434 webserver.cc:460] Webserver started at http://127.9.194.194:40431/ using document root <none> and password file <none>
I20260807 22:53:03.647859 23434 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:03.648124 23434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:03.648595 23434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:53:03 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143183, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:03.675619 23434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "ce38ac8cf0d047d9aacf6dedabd5579a"
format_stamp: "Formatted at 2026-08-07 22:53:03 on dist-test-slave-5k54"
server_key: "09ce607734a7b140a598dbe04c871653"
server_key_iv: "d322d21ed719bde946c873425287a715"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:03.677949 23434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "ce38ac8cf0d047d9aacf6dedabd5579a"
format_stamp: "Formatted at 2026-08-07 22:53:03 on dist-test-slave-5k54"
server_key: "09ce607734a7b140a598dbe04c871653"
server_key_iv: "d322d21ed719bde946c873425287a715"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:03.696218 23434 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.012s sys 0.004s
I20260807 22:53:03.745594 23452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:03.747646 23434 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.006s sys 0.000s
I20260807 22:53:03.747916 23434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "ce38ac8cf0d047d9aacf6dedabd5579a"
format_stamp: "Formatted at 2026-08-07 22:53:03 on dist-test-slave-5k54"
server_key: "09ce607734a7b140a598dbe04c871653"
server_key_iv: "d322d21ed719bde946c873425287a715"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:03.748242 23434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:03.833892 23434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:03.839704 23434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:03.840122 23434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:03.851909 23434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:03.864468 23434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:03.864650 23434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 22:53:03.864866 23434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:03.864972 23434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:03.924068 23434 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:44429
I20260807 22:53:03.924098 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:44429 every 8 connection(s)
I20260807 22:53:03.925861 23434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:03.930930 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23434
I20260807 22:53:03.931109 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:53:03.931833 9995 external_mini_cluster.cc:1428] Setting key 23e44a5d1e8d9b6a8fb2f1ca66ad3c79
Aug 07 22:53:03 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143183, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
I20260807 22:53:03.972658 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:03.930155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:38201 (local address 127.9.194.254:43673)
0807 22:53:03.949208 (+ 19053us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:03.949215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:03.951660 (+ 2445us) server_negotiation.cc:407] Connection header received
0807 22:53:03.953085 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:03.953091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:03.953193 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:03.953275 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:03.955628 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:03.956714 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:03.959361 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:03.960018 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:03.960129 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:03.964897 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:03.964905 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:03.964910 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:03.964952 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:03.968549 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:03.969530 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:03.969535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:03.969538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:03.969683 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:03.970368 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:03.970374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:03.970378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:03.970748 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:03.970915 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:03.971924 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0807 22:53:03.972205 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18914,"thread_start_us":18790,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
I20260807 22:53:03.973898 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:03.929532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.194:38201)
0807 22:53:03.949825 (+ 20293us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:03.949892 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:03.952728 (+ 2836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:03.953838 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:03.953861 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:03.954505 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:03.955404 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:03.955453 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:03.956933 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:03.956939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:03.959172 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:03.959187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:03.960479 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:03.960487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:03.960768 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:03.961608 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:03.961648 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:03.964585 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:03.968747 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:03.968756 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:03.968773 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:03.969387 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:03.969883 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:03.969888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:03.969892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:03.970191 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:03.970905 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:03.970913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:03.971375 (+ 462us) client_negotiation.cc:769] Sending connection context
0807 22:53:03.971784 (+ 409us) client_negotiation.cc:240] Negotiation successful
0807 22:53:03.972398 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19966,"thread_start_us":19738,"threads_started":1}
I20260807 22:53:03.975939 23584 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:03.976260 23584 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:03.977099 23584 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:03.977134 23426 heartbeater.cc:503] Master 127.9.194.254:43673 was elected leader, sending a full tablet report...
I20260807 22:53:03.979935 23196 ts_manager.cc:194] Registered new tserver with Master: ce38ac8cf0d047d9aacf6dedabd5579a (127.9.194.194:44429)
I20260807 22:53:03.981000 23196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:38201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:53:04.015020 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:04.391036 23593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:04.391314 23593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:04.391410 23593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:04.398837 23593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:04.398991 23593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:04.399132 23593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:04.409022 23593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:04.410249 23593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:04.411552 23593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:04.424963 23598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:04.425076 23601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:04.425052 23599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:04.425627 23593 server_base.cc:1034] running on GCE node
I20260807 22:53:04.426268 23593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:04.427574 23593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:04.428782 23593 hybrid_clock.cc:648] HybridClock initialized: now 1786143184428722 us; error 51 us; skew 500 ppm
I20260807 22:53:04.429113 23593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:04.432503 23593 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:04.434032 23593 webserver.cc:460] Webserver started at http://127.9.194.195:40965/ using document root <none> and password file <none>
I20260807 22:53:04.434719 23593 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:04.434927 23593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:04.435250 23593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:04.450735 23593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "688b6a66e1624037b3e835336a6548b2"
format_stamp: "Formatted at 2026-08-07 22:53:04 on dist-test-slave-5k54"
server_key: "a2d3f5285c665ea1125359180f4ee307"
server_key_iv: "488b645d733b87a6b72e0ad99d70921d"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:04.451666 23593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "688b6a66e1624037b3e835336a6548b2"
format_stamp: "Formatted at 2026-08-07 22:53:04 on dist-test-slave-5k54"
server_key: "a2d3f5285c665ea1125359180f4ee307"
server_key_iv: "488b645d733b87a6b72e0ad99d70921d"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:04.457263 23593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:53:04.471967 23609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:04.473778 23593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 22:53:04.473976 23593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "688b6a66e1624037b3e835336a6548b2"
format_stamp: "Formatted at 2026-08-07 22:53:04 on dist-test-slave-5k54"
server_key: "a2d3f5285c665ea1125359180f4ee307"
server_key_iv: "488b645d733b87a6b72e0ad99d70921d"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:04.474239 23593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:04.497745 23593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:04.501631 23593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:04.501891 23593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:04.502893 23593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:04.504241 23593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:04.504319 23593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:04.504400 23593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:04.504446 23593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:04.539820 23593 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42519
I20260807 22:53:04.539866 23735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42519 every 8 connection(s)
I20260807 22:53:04.541198 23593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:04.549312 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23593
I20260807 22:53:04.549525 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:53:04.549969 9995 external_mini_cluster.cc:1428] Setting key 88f9df02764c748b387973322564c92d
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:04.562572 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:04.543994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:52895 (local address 127.9.194.254:43673)
0807 22:53:04.544496 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:04.544501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:04.545447 (+ 946us) server_negotiation.cc:407] Connection header received
0807 22:53:04.547272 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:04.547277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:04.547384 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:04.547525 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:04.548984 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.549855 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.551828 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.552520 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.552653 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:04.556425 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:04.556430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:04.556433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:04.556484 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:04.559504 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.560226 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.560232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.560236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.560371 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.560807 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.560813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.560816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.561119 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:04.561304 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:04.561957 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 22:53:04.562195 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":232,"threads_started":1}
I20260807 22:53:04.563637 23739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:04.544535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.195:52895)
0807 22:53:04.545276 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:04.545318 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:04.546998 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:04.547769 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:04.547778 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:04.548216 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:04.548800 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:04.548828 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.550011 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.550019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:04.551679 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:04.551699 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.552767 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.552775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:04.553058 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:04.553775 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:04.553796 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:04.556204 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:04.559669 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:04.559676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:04.559688 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:04.560098 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:04.560483 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:04.560498 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:04.560501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:04.560694 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:04.561261 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:04.561269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:04.561614 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 22:53:04.561882 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 22:53:04.562395 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":312,"threads_started":1}
I20260807 22:53:04.565320 23736 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:04.565690 23736 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:04.566411 23736 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:04.567885 23196 ts_manager.cc:194] Registered new tserver with Master: 688b6a66e1624037b3e835336a6548b2 (127.9.194.195:42519)
I20260807 22:53:04.568696 23196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:52895
I20260807 22:53:04.578063 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:53:04.579802 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23284
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:53:04.597100 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:04.580798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.9.194.193:39607)
0807 22:53:04.581451 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:04.581457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:04.581482 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:53:04.581543 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:04.581548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:04.581746 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:04.581868 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:04.583577 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.584584 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.586350 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.586926 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.587029 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:04.591774 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:04.591792 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:04.591814 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:04.591864 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:04.594644 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.595329 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.595335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.595337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.595545 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.595868 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.595871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.595873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.596186 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:04.596424 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:04.596630 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 22:53:04.596820 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":340,"threads_started":1}
I20260807 22:53:04.598393 23421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:04.683947 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23284
I20260807 22:53:04.701462 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23434
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:53:04.722967 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:04.702449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.9.194.194:44429)
0807 22:53:04.703003 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:04.703011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:04.703037 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:53:04.703114 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:04.703119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:04.703324 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:04.703540 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:04.704681 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.705671 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.706910 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.707519 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.707644 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:04.717068 (+ 9424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:04.717081 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:04.717095 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:04.717131 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:04.720049 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.720764 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.720771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.720773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.720917 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.721295 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.721300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.721303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.721661 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:04.721986 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:04.722326 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:53:04.722607 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":283,"threads_started":1}
I20260807 22:53:04.724326 23579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:04.808027 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23434
I20260807 22:53:04.826221 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23593
Aug 07 22:53:04 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:53:04.843210 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:04.827470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.9.194.195:42519)
0807 22:53:04.828167 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:04.828174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:04.828203 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:53:04.828263 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:04.828268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:04.828531 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:04.828662 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:04.829913 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.830931 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.832227 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:04.832804 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:04.832907 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:04.837407 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:04.837418 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:04.837432 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:04.837465 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:04.840393 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.841098 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.841105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.841108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.841232 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:04.841624 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:04.841629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:04.841631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:04.841958 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:04.842309 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:04.842593 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 22:53:04.842857 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":381,"threads_started":1}
I20260807 22:53:04.844563 23731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:04.931162 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23593
I20260807 22:53:04.949783 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23161
I20260807 22:53:04.951115 23231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:05.058565 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23161
I20260807 22:53:05.082762 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43673
--webserver_interface=127.9.194.254
--webserver_port=45955
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:43673
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:05.411023 23749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:05.411293 23749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:05.411486 23749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:05.418661 23749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:05.418783 23749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:05.418864 23749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:05.418972 23749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:05.419063 23749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:05.429188 23749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:43673
--ranger_config_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:43673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=45955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:05.430469 23749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:05.431871 23749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:05.444031 23755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:05.444060 23756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:05.444101 23758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:05.444984 23749 server_base.cc:1034] running on GCE node
I20260807 22:53:05.445749 23749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:05.447217 23749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:05.448441 23749 hybrid_clock.cc:648] HybridClock initialized: now 1786143185448359 us; error 68 us; skew 500 ppm
I20260807 22:53:05.448833 23749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:05 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:05.452164 23749 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:05.453616 23749 webserver.cc:460] Webserver started at http://127.9.194.254:45955/ using document root <none> and password file <none>
I20260807 22:53:05.454288 23749 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:05.454438 23749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:53:05 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:05.541260 23749 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:53:05.544435 23766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:05.545845 23749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:53:05.545987 23749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d835294dfe084b7e90ca1df886512f1b"
format_stamp: "Formatted at 2026-08-07 22:53:01 on dist-test-slave-5k54"
server_key: "88e37aca7270251a2d3bc5109370b7f7"
server_key_iv: "5335b5aa34eff0468bbf7ab5ee83d9b2"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:05.546924 23749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:05.562958 23749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:05.566412 23749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:05.566653 23749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:05.584578 23749 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:43673
I20260807 22:53:05.584600 23823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:43673 every 8 connection(s)
I20260807 22:53:05.586155 23749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:05.589619 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23749
I20260807 22:53:05.590698 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:39607
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:43673
--webserver_port=37865
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:05.594841 23824 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:53:05.600165 23824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Bootstrap starting.
I20260807 22:53:05.619308 23824 log.cc:824] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:05.628328 23824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:53:05.628989 23824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Bootstrap complete.
I20260807 22:53:05.638262 23824 raft_consensus.cc:348] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:05.639117 23824 raft_consensus.cc:729] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835294dfe084b7e90ca1df886512f1b, State: Initialized, Role: FOLLOWER
I20260807 22:53:05.639806 23824 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:05.640028 23824 raft_consensus.cc:388] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:05.640136 23824 raft_consensus.cc:482] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:05.640319 23824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:53:05.644157 23824 raft_consensus.cc:504] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:05.644630 23824 leader_election.cc:302] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d835294dfe084b7e90ca1df886512f1b; no voters:
I20260807 22:53:05.645275 23824 leader_election.cc:288] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:53:05.645408 23830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:53:05.646675 23830 raft_consensus.cc:686] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [term 2 LEADER]: Becoming Leader. State: Replica: d835294dfe084b7e90ca1df886512f1b, State: Running, Role: LEADER
I20260807 22:53:05.647310 23830 consensus_queue.cc:237] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } }
I20260807 22:53:05.647605 23824 sys_catalog.cc:564] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:05.649837 23832 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835294dfe084b7e90ca1df886512f1b. Latest consensus state: current_term: 2 leader_uuid: "d835294dfe084b7e90ca1df886512f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } } }
I20260807 22:53:05.650259 23832 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:05.650810 23831 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d835294dfe084b7e90ca1df886512f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835294dfe084b7e90ca1df886512f1b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 43673 } } }
I20260807 22:53:05.651090 23831 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:05.661801 23836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:05.669051 23836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:05.670926 23836 catalog_manager.cc:1259] Loaded cluster ID: 00bfaaea59054a76a0736161e0371ecc
I20260807 22:53:05.671118 23836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:05.676515 23836 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:05.678500 23836 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Loaded TSK: 0
I20260807 22:53:05.680516 23836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:53:05.684434 23824 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 22:53:06.050931 23827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:06.051321 23827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:06.051542 23827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:06.063745 23827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:06.063915 23827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:06.064124 23827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:06.080178 23827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:39607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=37865
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:06.081959 23827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:06.083676 23827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:06.154148 23866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:06.160172 23865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:06.161631 23868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:06.162200 23827 server_base.cc:1034] running on GCE node
I20260807 22:53:06.163007 23827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:06.170428 23827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:06.171659 23827 hybrid_clock.cc:648] HybridClock initialized: now 1786143186171582 us; error 75 us; skew 500 ppm
I20260807 22:53:06.172104 23827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:06.176590 23827 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:06.189100 23827 webserver.cc:460] Webserver started at http://127.9.194.193:37865/ using document root <none> and password file <none>
I20260807 22:53:06.190132 23827 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:06.190310 23827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:06.253867 23827 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.007s sys 0.012s
I20260807 22:53:06.262075 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:06.264117 23827 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.005s sys 0.001s
I20260807 22:53:06.264300 23827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "cb6338c7953647a3a08f6929f35f6d63"
format_stamp: "Formatted at 2026-08-07 22:53:02 on dist-test-slave-5k54"
server_key: "95a441baf5f9ff3938df4f04c0fedd08"
server_key_iv: "91e6ebe2a0d04285139c8d3cf0db7443"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:06.265792 23827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:06.296689 23827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:06.301128 23827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:06.301483 23827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:06.305166 23827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:06.308367 23827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:06.308491 23827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:06.308596 23827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:06.308672 23827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:06.357290 23827 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39607
I20260807 22:53:06.357337 24000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39607 every 8 connection(s)
I20260807 22:53:06.358786 23827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:06.360189 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 23827
I20260807 22:53:06.361054 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:44429
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:43673
--webserver_port=40431
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:06.403121 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:06.362944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.193:58953)
0807 22:53:06.374534 (+ 11590us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:06.374587 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:06.377499 (+ 2912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:06.380187 (+ 2688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:06.380197 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:06.380900 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:06.381686 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:06.381721 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:06.384373 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:06.384381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:06.386679 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:06.386700 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:06.388688 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:06.388697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:06.389031 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:06.389901 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:06.389937 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:06.393082 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:06.397745 (+ 4663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:06.397755 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:06.397772 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:06.398393 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:06.398959 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:06.398965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:06.398969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:06.399224 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:06.400347 (+ 1123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:06.400355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:06.400791 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 22:53:06.401109 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 22:53:06.401706 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11257,"thread_start_us":10990,"threads_started":1}
I20260807 22:53:06.405546 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:06.364445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58953 (local address 127.9.194.254:43673)
0807 22:53:06.373875 (+ 9430us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:06.373896 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:06.374827 (+ 931us) server_negotiation.cc:407] Connection header received
0807 22:53:06.378324 (+ 3497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:06.378346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:06.378892 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:06.379321 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:06.381966 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:06.384156 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:06.386880 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:06.388083 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:06.388281 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:06.393385 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:06.393397 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:06.393417 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:06.393473 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:06.397549 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:06.398568 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:06.398578 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:06.398586 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:06.398800 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:06.399366 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:06.399372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:06.399376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:06.400068 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:06.400259 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:06.403859 (+ 3600us) server_negotiation.cc:299] Negotiation successful
0807 22:53:06.404281 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9089,"thread_start_us":8807,"threads_started":1}
I20260807 22:53:06.409920 24002 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:06.410249 24002 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:06.411126 24002 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
W20260807 22:53:06.414409 24002 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:43673 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:53:06.414620 24002 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:06.414880 24002 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:06.416045 24002 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:06.416355 24002 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
W20260807 22:53:06.417507 24002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:53:06.768978 24006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:06.769356 24006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:06.769569 24006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:06.782413 24006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:06.782636 24006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:06.782891 24006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:06.798976 24006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:44429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=40431
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:06.800904 24006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:06.802618 24006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:06.844045 24022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:06.845369 24023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:06.846827 24025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:06.848012 24006 server_base.cc:1034] running on GCE node
I20260807 22:53:06.848977 24006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:06.850227 24006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:06.851583 24006 hybrid_clock.cc:648] HybridClock initialized: now 1786143186851508 us; error 68 us; skew 500 ppm
I20260807 22:53:06.852011 24006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:06.855356 24006 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:06.858351 24006 webserver.cc:460] Webserver started at http://127.9.194.194:40431/ using document root <none> and password file <none>
I20260807 22:53:06.859115 24006 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:06.859285 24006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
Aug 07 22:53:06 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:06.921473 24006 fs_manager.cc:709] Time spent opening directory manager: real 0.028s user 0.008s sys 0.006s
I20260807 22:53:06.932716 24033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:06.941713 24006 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.004s sys 0.000s
I20260807 22:53:06.941932 24006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "ce38ac8cf0d047d9aacf6dedabd5579a"
format_stamp: "Formatted at 2026-08-07 22:53:03 on dist-test-slave-5k54"
server_key: "09ce607734a7b140a598dbe04c871653"
server_key_iv: "d322d21ed719bde946c873425287a715"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:06.943460 24006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:06.986377 24006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:06.991899 24006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:06.992296 24006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:07.011907 24006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:07.020224 24006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:07.020457 24006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 22:53:07.020670 24006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:07.020823 24006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:07.131781 24006 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:44429
I20260807 22:53:07.131839 24153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:44429 every 8 connection(s)
I20260807 22:53:07.133823 24006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:07.135632 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24006
I20260807 22:53:07.136449 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.194.212:42389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:42519
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:43673
--webserver_port=40965
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:07 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:07.189143 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:07.143699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:53507 (local address 127.9.194.254:43673)
0807 22:53:07.148331 (+ 4632us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:07.148339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:07.160378 (+ 12039us) server_negotiation.cc:407] Connection header received
0807 22:53:07.163748 (+ 3370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:07.163758 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:07.163880 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:07.163972 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:07.166657 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:07.168021 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:07.171663 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:07.172477 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:07.172627 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:07.181218 (+ 8591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:07.181226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:07.181231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:07.181277 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:07.184913 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:07.185991 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:07.185997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:07.186001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:07.186149 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:07.186723 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:07.186730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:07.186735 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:07.187105 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:07.187231 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:07.188516 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0807 22:53:07.188751 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4214,"thread_start_us":3956,"threads_started":1}
I20260807 22:53:07.191538 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:07.143087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.194:53507)
0807 22:53:07.160174 (+ 17087us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:07.160232 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:07.163318 (+ 3086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:07.164631 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:07.164645 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:07.165429 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:07.166402 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:07.166436 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:07.168274 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:07.168282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:07.171462 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:07.171483 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:07.173080 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:07.173091 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:07.173475 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:07.177984 (+ 4509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:07.178018 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:07.180906 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:07.185209 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:07.185220 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:07.185241 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:07.185841 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:07.186302 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:07.186307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:07.186310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:07.186590 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:07.187330 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:07.187337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:07.188078 (+ 741us) client_negotiation.cc:769] Sending connection context
0807 22:53:07.189716 (+ 1638us) client_negotiation.cc:240] Negotiation successful
0807 22:53:07.190265 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16748,"thread_start_us":16515,"threads_started":1}
I20260807 22:53:07.193495 24154 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:07.193836 24154 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:07.194576 24154 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:07.196557 23785 ts_manager.cc:194] Registered new tserver with Master: ce38ac8cf0d047d9aacf6dedabd5579a (127.9.194.194:44429)
I20260807 22:53:07.198709 23785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:53507
I20260807 22:53:07.417696 24002 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:07.418172 24002 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:07.419652 23785 ts_manager.cc:194] Registered new tserver with Master: cb6338c7953647a3a08f6929f35f6d63 (127.9.194.193:39607)
I20260807 22:53:07.420907 23785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:58953
Aug 07 22:53:07 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
W20260807 22:53:07.613814 24157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:07.614326 24157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:07.614571 24157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:07.627705 24157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:07.627942 24157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:07.628192 24157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:07.645840 24157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.194.212:42389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:42519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=40965
--tserver_master_addrs=127.9.194.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:07.648094 24157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:07.650133 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:07.667069 24163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:07.668731 24164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:07.674010 24157 server_base.cc:1034] running on GCE node
W20260807 22:53:07.674401 24166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:07.675181 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:07.676662 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:07.677971 24157 hybrid_clock.cc:648] HybridClock initialized: now 1786143187677896 us; error 62 us; skew 500 ppm
I20260807 22:53:07.678359 24157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:07 dist-test-slave-5k54 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143187, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:07.682722 24157 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:07.684932 24157 webserver.cc:460] Webserver started at http://127.9.194.195:40965/ using document root <none> and password file <none>
I20260807 22:53:07.685899 24157 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:07.686062 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:53:07 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143187, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:07.712423 24157 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 22:53:07.720085 24174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:07.722021 24157 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.005s
I20260807 22:53:07.722196 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "688b6a66e1624037b3e835336a6548b2"
format_stamp: "Formatted at 2026-08-07 22:53:04 on dist-test-slave-5k54"
server_key: "a2d3f5285c665ea1125359180f4ee307"
server_key_iv: "488b645d733b87a6b72e0ad99d70921d"
server_key_version: "kuduclusterkey@0"
I20260807 22:53:07.723695 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:07.757053 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:07.762676 24157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:07.763084 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:07.765233 24157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:07.767160 24157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:07.767324 24157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:07.767504 24157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:07.767643 24157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:53:07 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143186, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for HTTP/127.9.194.212@KRBTEST.COM
I20260807 22:53:08.029664 24157 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42519
I20260807 22:53:08.029711 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42519 every 8 connection(s)
I20260807 22:53:08.031491 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:08.035893 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24157
Aug 07 22:53:08 dist-test-slave-5k54 krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143187, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:08.087548 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.037694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.9.194.254:43673)
0807 22:53:08.056612 (+ 18918us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.056621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.061435 (+ 4814us) server_negotiation.cc:407] Connection header received
0807 22:53:08.061650 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.061659 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.061769 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.061920 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.065037 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.066617 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.075692 (+ 9075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.076442 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.076571 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.080786 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.080794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.080799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.080856 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:08.084308 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.085027 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.085033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.085036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.085167 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.085717 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.085721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.085723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.086136 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:08.086348 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.086808 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.087103 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18747,"thread_start_us":18930,"threads_started":1}
I20260807 22:53:08.093129 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.045614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:48335 (local address 127.9.194.254:43673)
0807 22:53:08.058135 (+ 12521us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.058142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.060832 (+ 2690us) server_negotiation.cc:407] Connection header received
0807 22:53:08.070994 (+ 10162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.071001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.071103 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.071188 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.075749 (+ 4561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.077125 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.079639 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.080282 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.080395 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.086930 (+ 6535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.086937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.086941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.086978 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:08.089969 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.090816 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.090825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.090828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.090942 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.091406 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.091412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.091419 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.091707 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:08.091857 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.092555 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.092755 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12383,"thread_start_us":12227,"threads_started":1}
I20260807 22:53:08.094316 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.044930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:43673 (local address 127.9.194.195:48335)
0807 22:53:08.058799 (+ 13869us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.058857 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.062491 (+ 3634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.072034 (+ 9543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.072048 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.072785 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:08.073685 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.073713 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.077331 (+ 3618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.077349 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.079467 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.079483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.080753 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.080764 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.081104 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.083789 (+ 2685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:08.083821 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:08.086622 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:08.090110 (+ 3488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:08.090119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:08.090137 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:08.090686 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:08.091056 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:08.091061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:08.091064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:08.091278 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:08.091844 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:08.091852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:08.092193 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.092454 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.093055 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13518,"thread_start_us":13248,"threads_started":1}
I20260807 22:53:08.095789 24297 heartbeater.cc:347] Connected to a master server at 127.9.194.254:43673
I20260807 22:53:08.096118 24297 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:08.096814 24297 heartbeater.cc:511] Master 127.9.194.254:43673 requested a full tablet report, sending...
I20260807 22:53:08.098210 23785 ts_manager.cc:194] Registered new tserver with Master: 688b6a66e1624037b3e835336a6548b2 (127.9.194.195:42519)
I20260807 22:53:08.099181 23785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:48335
I20260807 22:53:08.102937 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:53:08.118999 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.106005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.9.194.254:43673)
0807 22:53:08.106162 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.106166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.106692 (+ 526us) server_negotiation.cc:407] Connection header received
0807 22:53:08.106887 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.106893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.106971 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.107073 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.108641 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.109349 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.110698 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.111173 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.111302 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.113870 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.113876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.113881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.113922 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:08.116395 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.117054 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.117061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.117065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.117198 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.117599 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.117605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.117608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.117877 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:08.118143 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.118440 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.118666 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:53:08.123817 23785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:53:08.131413 23785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:53:08.168722 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.145823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42519 (local address 127.0.0.1:44346)
0807 22:53:08.146820 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.146878 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.147076 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.159683 (+ 12607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.159695 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.159743 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:08.160306 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.160322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.162946 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.162952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.165621 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.165636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.167693 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.167702 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.167911 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.167978 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.168115 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.168327 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":1194,"threads_started":1}
I20260807 22:53:08.169065 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.146141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44346 (local address 127.9.194.195:42519)
0807 22:53:08.158750 (+ 12609us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.158762 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.158807 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:53:08.158948 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.158960 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.159318 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.160597 (+ 1279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:08.161013 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.162791 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.165837 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.167395 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.167618 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.167850 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.168497 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.168668 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12389,"thread_start_us":12207,"threads_started":1}
I20260807 22:53:08.169800 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.146497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58988 (local address 127.9.194.193:39607)
0807 22:53:08.157871 (+ 11374us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.157885 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.159216 (+ 1331us) server_negotiation.cc:407] Connection header received
0807 22:53:08.159569 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.159578 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.159835 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.160161 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:08.162207 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.163839 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.166466 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.167981 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.168145 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.168385 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.169113 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.169484 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11121,"thread_start_us":11399,"threads_started":1}
I20260807 22:53:08.170476 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.145887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39607 (local address 127.0.0.1:58988)
0807 22:53:08.159072 (+ 13185us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.159092 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.159331 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.160528 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.160536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.160572 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:08.161208 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.161226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.164046 (+ 2820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.164052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.166273 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.166287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.168212 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.168219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.168423 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.168457 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.168549 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.168674 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13045,"mutex_wait_us":1,"thread_start_us":12907,"threads_started":1}
I20260807 22:53:08.170884 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.146625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44429 (local address 127.0.0.1:37286)
0807 22:53:08.161823 (+ 15198us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.161845 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.162363 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.163495 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.163500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.163530 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:08.163959 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.163971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.165700 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.165705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.167626 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.167637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.169519 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.169525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.169704 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.169760 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.170231 (+ 471us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.170367 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15071,"thread_start_us":14906,"threads_started":1}
I20260807 22:53:08.171192 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.147194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.9.194.194:44429)
0807 22:53:08.162204 (+ 15010us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.162212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.162241 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:53:08.162576 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.162584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.162830 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.163240 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:08.164204 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.165500 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.167807 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.169266 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.169449 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.169688 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.170592 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.170767 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14834,"thread_start_us":14680,"threads_started":1}
I20260807 22:53:08.173471 24225 tablet_service.cc:1467] Processing CreateTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed (DEFAULT_TABLE table=test-table [id=8190688bf8c84e9ea3b6bc0888b22156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:08.174105 23932 tablet_service.cc:1467] Processing CreateTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed (DEFAULT_TABLE table=test-table [id=8190688bf8c84e9ea3b6bc0888b22156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:08.175061 24225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8be0c3ade88b4d659d4b04fb220ec4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:08.174791 24084 tablet_service.cc:1467] Processing CreateTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed (DEFAULT_TABLE table=test-table [id=8190688bf8c84e9ea3b6bc0888b22156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:08.175515 23932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8be0c3ade88b4d659d4b04fb220ec4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:08.176167 24084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8be0c3ade88b4d659d4b04fb220ec4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:08.188460 24327 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Bootstrap starting.
I20260807 22:53:08.191242 24328 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Bootstrap starting.
I20260807 22:53:08.192283 24327 tablet_bootstrap.cc:654] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:08.193152 24329 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Bootstrap starting.
I20260807 22:53:08.194314 24327 log.cc:824] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:08.195926 24328 tablet_bootstrap.cc:654] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:08.196450 24329 tablet_bootstrap.cc:654] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:08.197188 24328 log.cc:824] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:08.197448 24329 log.cc:824] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:08.197748 24327 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: No bootstrap required, opened a new log
I20260807 22:53:08.198089 24327 ts_tablet_manager.cc:1397] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 22:53:08.200414 24328 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: No bootstrap required, opened a new log
I20260807 22:53:08.200496 24329 tablet_bootstrap.cc:492] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: No bootstrap required, opened a new log
I20260807 22:53:08.200682 24328 ts_tablet_manager.cc:1397] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 22:53:08.200704 24329 ts_tablet_manager.cc:1397] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:53:08.202723 24154 heartbeater.cc:503] Master 127.9.194.254:43673 was elected leader, sending a full tablet report...
I20260807 22:53:08.207558 24327 raft_consensus.cc:348] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.207943 24327 raft_consensus.cc:374] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:08.208108 24327 raft_consensus.cc:729] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6338c7953647a3a08f6929f35f6d63, State: Initialized, Role: FOLLOWER
I20260807 22:53:08.208714 24327 consensus_queue.cc:260] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.209794 24002 heartbeater.cc:503] Master 127.9.194.254:43673 was elected leader, sending a full tablet report...
I20260807 22:53:08.211410 24328 raft_consensus.cc:348] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.211879 24328 raft_consensus.cc:374] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:08.211978 24328 raft_consensus.cc:729] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce38ac8cf0d047d9aacf6dedabd5579a, State: Initialized, Role: FOLLOWER
I20260807 22:53:08.212001 24327 ts_tablet_manager.cc:1428] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 22:53:08.212252 24329 raft_consensus.cc:348] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.212664 24329 raft_consensus.cc:374] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:08.212764 24329 raft_consensus.cc:729] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688b6a66e1624037b3e835336a6548b2, State: Initialized, Role: FOLLOWER
I20260807 22:53:08.212662 24328 consensus_queue.cc:260] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.213409 24329 consensus_queue.cc:260] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.214668 24328 ts_tablet_manager.cc:1428] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 22:53:08.215615 24297 heartbeater.cc:503] Master 127.9.194.254:43673 was elected leader, sending a full tablet report...
I20260807 22:53:08.216295 24329 ts_tablet_manager.cc:1428] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260807 22:53:08.325991 24298 tablet.cc:2369] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:53:08.381000 24003 tablet.cc:2369] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:53:08.401566 24158 tablet.cc:2369] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:08.582901 24334 raft_consensus.cc:482] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:08.583310 24334 raft_consensus.cc:504] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.585726 24334 leader_election.cc:288] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 688b6a66e1624037b3e835336a6548b2 (127.9.194.195:42519), ce38ac8cf0d047d9aacf6dedabd5579a (127.9.194.194:44429)
I20260807 22:53:08.594409 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.586838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:45749 (local address 127.9.194.195:42519)
0807 22:53:08.587367 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.587373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.587394 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:53:08.587497 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.587503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.587573 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.587657 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:08.588459 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.589626 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.592579 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.593366 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.593478 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.593531 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.594018 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.594149 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364}
I20260807 22:53:08.594516 24340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.585915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42519 (local address 127.9.194.193:45749)
0807 22:53:08.586748 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.586773 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.586950 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.587826 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.587832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.587861 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:08.588249 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.588267 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.589834 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.589842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.592367 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.592385 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.593698 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.593702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.593839 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.593861 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.593968 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.594108 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"threads_started":1}
I20260807 22:53:08.597539 24248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8be0c3ade88b4d659d4b04fb220ec4ed" candidate_uuid: "cb6338c7953647a3a08f6929f35f6d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688b6a66e1624037b3e835336a6548b2" is_pre_election: true
I20260807 22:53:08.598043 24248 raft_consensus.cc:2454] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6338c7953647a3a08f6929f35f6d63 in term 0.
I20260807 22:53:08.598747 23880 leader_election.cc:302] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688b6a66e1624037b3e835336a6548b2, cb6338c7953647a3a08f6929f35f6d63; no voters:
I20260807 22:53:08.599188 24334 raft_consensus.cc:2781] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:53:08.599332 24334 raft_consensus.cc:482] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:53:08.599443 24334 raft_consensus.cc:3037] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:08.600296 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.586505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:56195 (local address 127.9.194.194:44429)
0807 22:53:08.586695 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.586702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.594882 (+ 8180us) server_negotiation.cc:407] Connection header received
0807 22:53:08.595209 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.595215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.595300 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.595401 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:08.595800 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.596542 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.598173 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.599364 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.599566 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.599644 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.599926 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.600069 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:53:08.600322 24340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.586024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:44429 (local address 127.9.194.193:56195)
0807 22:53:08.594746 (+ 8722us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:08.594766 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:08.594988 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:08.595482 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:08.595487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:08.595513 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:08.595659 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.595666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.596683 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.596691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.598025 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:08.598038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.599551 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.599557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:08.599758 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.599786 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 22:53:08.599889 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:53:08.600019 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8576,"thread_start_us":338,"threads_started":1}
I20260807 22:53:08.601239 24106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8be0c3ade88b4d659d4b04fb220ec4ed" candidate_uuid: "cb6338c7953647a3a08f6929f35f6d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" is_pre_election: true
I20260807 22:53:08.601661 24106 raft_consensus.cc:2454] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6338c7953647a3a08f6929f35f6d63 in term 0.
I20260807 22:53:08.602684 24334 raft_consensus.cc:504] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.603896 24334 leader_election.cc:288] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [CANDIDATE]: Term 1 election: Requested vote from peers 688b6a66e1624037b3e835336a6548b2 (127.9.194.195:42519), ce38ac8cf0d047d9aacf6dedabd5579a (127.9.194.194:44429)
I20260807 22:53:08.603873 24106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8be0c3ade88b4d659d4b04fb220ec4ed" candidate_uuid: "cb6338c7953647a3a08f6929f35f6d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a"
I20260807 22:53:08.603860 24248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8be0c3ade88b4d659d4b04fb220ec4ed" candidate_uuid: "cb6338c7953647a3a08f6929f35f6d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688b6a66e1624037b3e835336a6548b2"
I20260807 22:53:08.604094 24248 raft_consensus.cc:3037] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:08.604063 24106 raft_consensus.cc:3037] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:08.607317 24106 raft_consensus.cc:2454] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6338c7953647a3a08f6929f35f6d63 in term 1.
I20260807 22:53:08.607317 24248 raft_consensus.cc:2454] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6338c7953647a3a08f6929f35f6d63 in term 1.
I20260807 22:53:08.607749 23881 leader_election.cc:302] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb6338c7953647a3a08f6929f35f6d63, ce38ac8cf0d047d9aacf6dedabd5579a; no voters:
I20260807 22:53:08.608129 24334 raft_consensus.cc:2781] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:08.608773 24334 raft_consensus.cc:686] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 1 LEADER]: Becoming Leader. State: Replica: cb6338c7953647a3a08f6929f35f6d63, State: Running, Role: LEADER
I20260807 22:53:08.609367 24334 consensus_queue.cc:237] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } }
I20260807 22:53:08.614223 23785 catalog_manager.cc:5495] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb6338c7953647a3a08f6929f35f6d63 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "cb6338c7953647a3a08f6929f35f6d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb6338c7953647a3a08f6929f35f6d63" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 } health_report { overall_health: UNKNOWN } } }
I20260807 22:53:08.631583 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.625757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.9.194.193:39607)
0807 22:53:08.625980 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.625988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.626321 (+ 333us) server_negotiation.cc:407] Connection header received
0807 22:53:08.626585 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.626589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.626662 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.626811 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:53:08.627233 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.628011 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.629422 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.629830 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.629963 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.630445 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:53:08.630919 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:53:08.631081 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.631174 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.631252 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:53:08.639611 24248 raft_consensus.cc:1261] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 1 FOLLOWER]: Refusing update from remote peer cb6338c7953647a3a08f6929f35f6d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:08.639611 24106 raft_consensus.cc:1261] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 1 FOLLOWER]: Refusing update from remote peer cb6338c7953647a3a08f6929f35f6d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:08.640504 24341 consensus_queue.cc:1035] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688b6a66e1624037b3e835336a6548b2" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:08.640832 24334 consensus_queue.cc:1035] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce38ac8cf0d047d9aacf6dedabd5579a" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 44429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:08.665102 23784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:08.665432 23784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:08.669443 23784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d835294dfe084b7e90ca1df886512f1b: Sending DeleteTablet for 3 replicas of tablet 8be0c3ade88b4d659d4b04fb220ec4ed
I20260807 22:53:08.670696 24225 tablet_service.cc:1514] Processing DeleteTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:44346
I20260807 22:53:08.670773 23932 tablet_service.cc:1514] Processing DeleteTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:58988
I20260807 22:53:08.670749 24084 tablet_service.cc:1514] Processing DeleteTablet for tablet 8be0c3ade88b4d659d4b04fb220ec4ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37286
I20260807 22:53:08.673206 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23827
I20260807 22:53:08.704269 24356 tablet_replica.cc:330] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: stopping tablet replica
I20260807 22:53:08.704998 24356 raft_consensus.cc:2229] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:08.705734 24356 raft_consensus.cc:2258] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:08.710570 24356 ts_tablet_manager.cc:1905] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:08.711990 24357 tablet_replica.cc:330] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: stopping tablet replica
I20260807 22:53:08.712340 24358 tablet_replica.cc:330] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: stopping tablet replica
I20260807 22:53:08.712499 24357 raft_consensus.cc:2229] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:08.712873 24358 raft_consensus.cc:2229] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:53:08.713526 24358 raft_consensus.cc:2258] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:08.714661 24357 raft_consensus.cc:2258] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:08.717859 24357 ts_tablet_manager.cc:1905] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:08.721935 24356 ts_tablet_manager.cc:1918] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:08.722139 24356 log.cc:1196] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/8be0c3ade88b4d659d4b04fb220ec4ed
I20260807 22:53:08.722972 24356 ts_tablet_manager.cc:1939] T 8be0c3ade88b4d659d4b04fb220ec4ed P ce38ac8cf0d047d9aacf6dedabd5579a: Deleting consensus metadata
I20260807 22:53:08.723609 24358 ts_tablet_manager.cc:1905] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:08.725306 23768 catalog_manager.cc:4841] TS ce38ac8cf0d047d9aacf6dedabd5579a (127.9.194.194:44429): tablet 8be0c3ade88b4d659d4b04fb220ec4ed (table test-table [id=8190688bf8c84e9ea3b6bc0888b22156]) successfully deleted
I20260807 22:53:08.728153 24357 ts_tablet_manager.cc:1918] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:08.728420 24357 log.cc:1196] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/8be0c3ade88b4d659d4b04fb220ec4ed
I20260807 22:53:08.729602 24357 ts_tablet_manager.cc:1939] T 8be0c3ade88b4d659d4b04fb220ec4ed P 688b6a66e1624037b3e835336a6548b2: Deleting consensus metadata
I20260807 22:53:08.731938 23767 catalog_manager.cc:4841] TS 688b6a66e1624037b3e835336a6548b2 (127.9.194.195:42519): tablet 8be0c3ade88b4d659d4b04fb220ec4ed (table test-table [id=8190688bf8c84e9ea3b6bc0888b22156]) successfully deleted
I20260807 22:53:08.732160 24358 ts_tablet_manager.cc:1918] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:08.732456 24358 log.cc:1196] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/8be0c3ade88b4d659d4b04fb220ec4ed
I20260807 22:53:08.733023 24358 ts_tablet_manager.cc:1939] T 8be0c3ade88b4d659d4b04fb220ec4ed P cb6338c7953647a3a08f6929f35f6d63: Deleting consensus metadata
I20260807 22:53:08.734591 23769 catalog_manager.cc:4841] TS cb6338c7953647a3a08f6929f35f6d63 (127.9.194.193:39607): tablet 8be0c3ade88b4d659d4b04fb220ec4ed (table test-table [id=8190688bf8c84e9ea3b6bc0888b22156]) successfully deleted
I20260807 22:53:08.737744 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.675014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.9.194.193:39607)
0807 22:53:08.682876 (+ 7862us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.682886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.699707 (+ 16821us) server_negotiation.cc:407] Connection header received
0807 22:53:08.700659 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.700670 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.700790 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.700908 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.706490 (+ 5582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.708429 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.715397 (+ 6968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.716154 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.716268 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.729628 (+ 13360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.729644 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.729664 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.729734 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:08.734300 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.735267 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.735275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.735277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.735535 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.736093 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.736098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.736103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.736519 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:08.736767 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.737164 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.737380 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7679}
I20260807 22:53:08.738498 23996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:08.843681 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23827
I20260807 22:53:08.864416 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24006
I20260807 22:53:08.880395 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.865269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.9.194.194:44429)
0807 22:53:08.865477 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.865484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.865561 (+ 77us) server_negotiation.cc:407] Connection header received
0807 22:53:08.865770 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.865782 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.865851 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.865957 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.867361 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.868151 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.869875 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.870288 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.870424 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.874263 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.874280 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.874300 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.874369 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:08.877703 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.878338 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.878345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.878347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.878490 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:08.878815 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:08.878818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:08.878820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:08.879213 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:08.879713 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:08.879839 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 22:53:08.880050 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:53:08.881127 24149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:08.970826 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24006
I20260807 22:53:08.989732 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24157
I20260807 22:53:09.002409 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:08.990545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.9.194.195:42519)
0807 22:53:08.990715 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:08.990719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:08.990737 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:53:08.990867 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:08.990870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:08.990935 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:08.991021 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:08.992370 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.993161 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.994386 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:08.994767 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:08.994855 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:08.997414 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:08.997429 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:08.997445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:08.997477 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:09.000175 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:09.000806 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:09.000813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:09.000815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:09.000936 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:09.001226 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:09.001229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:09.001231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:09.001568 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:09.001777 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:09.001985 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 22:53:09.002141 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:53:09.003024 24292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:09.091955 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24157
I20260807 22:53:09.111733 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 23749
I20260807 22:53:09.112881 23818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:09.224731 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 23749
I20260807 22:53:09.250247 9995 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:53:09.595587 9995 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:53:09.595834 9995 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:53:09.940653 9995 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:53:09.940 UTC [22720] LOG: received smart shutdown request
2026-08-07 22:53:09.944 UTC [22720] LOG: background worker "logical replication launcher" (PID 22729) exited with exit code 1
2026-08-07 22:53:09.944 UTC [22724] LOG: shutting down
2026-08-07 22:53:10.106 UTC [22720] LOG: database system is shut down
2026-08-07T22:53:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54341 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:53:10 dist-test-slave-5k54 krb5kdc[24372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:53:10 dist-test-slave-5k54 krb5kdc[24372](info): set up 2 sockets
Aug 07 22:53:10 dist-test-slave-5k54 krb5kdc[24372](info): commencing operation
krb5kdc: starting...
W20260807 22:53:11.400772 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:53:11 dist-test-slave-5k54 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:53:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:53:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:53:11.668093 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38821
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:12.018297 24385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:12.018639 24385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:12.018801 24385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:12.026325 24385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:12.026472 24385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:12.026592 24385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:12.026705 24385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:12.026786 24385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:12.036957 24385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:12.038303 24385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:12.039711 24385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:12.051692 24391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:12.051771 24394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:12.051790 24392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:12.052567 24385 server_base.cc:1034] running on GCE node
I20260807 22:53:12.053496 24385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:12.054888 24385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:12.056119 24385 hybrid_clock.cc:648] HybridClock initialized: now 1786143192056062 us; error 55 us; skew 500 ppm
I20260807 22:53:12.056465 24385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:12.059695 24385 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:12.061244 24385 webserver.cc:460] Webserver started at http://127.9.194.254:33113/ using document root <none> and password file <none>
I20260807 22:53:12.061990 24385 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:12.062139 24385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:12.062455 24385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:12.065609 24385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "d3b953fb3773486c8231b405c23aef4c"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "1edc64256100a40a5992ef6ae05aa5b1"
server_key_iv: "dd45a6d39283c6bcaeb0a09df942809f"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.066424 24385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "d3b953fb3773486c8231b405c23aef4c"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "1edc64256100a40a5992ef6ae05aa5b1"
server_key_iv: "dd45a6d39283c6bcaeb0a09df942809f"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.071547 24385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:53:12.075035 24401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:12.076423 24385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:53:12.076606 24385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d3b953fb3773486c8231b405c23aef4c"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "1edc64256100a40a5992ef6ae05aa5b1"
server_key_iv: "dd45a6d39283c6bcaeb0a09df942809f"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.076804 24385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:12.092465 24385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:12.097196 24385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:12.097536 24385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:12.118280 24385 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:38821
I20260807 22:53:12.118305 24456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:38821 every 8 connection(s)
I20260807 22:53:12.119868 24385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:12.123796 24457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:12.124519 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24385
I20260807 22:53:12.124780 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:53:12.125639 9995 external_mini_cluster.cc:1428] Setting key 34f64e0f4b2a8e2073b8c540ca708f9b
I20260807 22:53:12.134107 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.194.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.253@KRBTEST.COM" created.
I20260807 22:53:12.158676 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.139193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.9.194.254:38821)
0807 22:53:12.139965 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:12.139976 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:12.140035 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:53:12.141300 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:12.141311 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:12.141619 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:12.141838 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:12.142973 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.145058 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.146431 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.147471 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.147666 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.151009 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:12.151022 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:12.151041 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:12.151084 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:12.154300 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.155002 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.155008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.155010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.155152 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.155642 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.155650 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.155655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.156148 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:12.156356 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:12.156865 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 22:53:12.157249 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":544,"thread_start_us":377,"threads_started":1}
I20260807 22:53:12.158713 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.137886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38821 (local address 127.0.0.1:42176)
0807 22:53:12.138725 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.138779 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:12.140896 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:12.141866 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:12.141871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:12.142430 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:12.142765 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.142781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.145210 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.145214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.146286 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.146298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.147688 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.147692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.147820 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.148502 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:12.148524 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:12.150756 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:12.154429 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.154439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.154448 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.154857 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.155258 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.155261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.155264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.155530 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.156293 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:12.156300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:12.156499 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 22:53:12.156749 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 22:53:12.157358 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":155,"thread_start_us":353,"threads_started":1}
I20260807 22:53:12.161900 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:53:12.162985 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.162592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54338)
0807 22:53:12.162757 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.162865 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 22:53:12.163270 24402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
I20260807 22:53:12.164893 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.164550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54344)
0807 22:53:12.164694 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.164759 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:53:12.165102 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:33391: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
I20260807 22:53:12.184754 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:33391
--webserver_interface=127.9.194.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:12.214500 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } attempt: 1
I20260807 22:53:12.215694 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.215183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54358)
0807 22:53:12.215387 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.215555 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 22:53:12.216686 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.216348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54360)
0807 22:53:12.216489 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.216560 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:53:12.216879 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:33391: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
I20260807 22:53:12.285133 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } attempt: 2
I20260807 22:53:12.286288 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.285790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54376)
0807 22:53:12.286058 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.286147 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 22:53:12.287292 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.286933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54388)
0807 22:53:12.287081 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.287169 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 22:53:12.287570 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:33391: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
I20260807 22:53:12.378829 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } attempt: 3
I20260807 22:53:12.380146 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.379702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54390)
0807 22:53:12.379899 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.380017 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 22:53:12.381315 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.380993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54400)
0807 22:53:12.381121 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.381187 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:53:12.381618 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:33391: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
W20260807 22:53:12.518671 24465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:12.518947 24465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:12.519042 24465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:53:12.524881 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } attempt: 4
I20260807 22:53:12.526088 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.525571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54404)
0807 22:53:12.525868 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.525957 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260807 22:53:12.526863 24465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:12.526978 24465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:12.527091 24465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:12.527192 24465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260807 22:53:12.527215 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.526777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54406)
0807 22:53:12.526898 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.527025 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:53:12.527274 24465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:53:12.527472 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:33391: Network error: Client connection negotiation failed: client connection to 127.9.194.253:33391: connect: Connection refused (error 111)
I20260807 22:53:12.537351 24465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.194.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:12.538748 24465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:12.540169 24465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:12.551901 24471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:12.551857 24473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:12.552095 24470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:12.552939 24465 server_base.cc:1034] running on GCE node
I20260807 22:53:12.553725 24465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:12.555127 24465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:12.556351 24465 hybrid_clock.cc:648] HybridClock initialized: now 1786143192556267 us; error 80 us; skew 500 ppm
I20260807 22:53:12.556703 24465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:12.559864 24465 init.cc:377] Logged in from keytab as kudu/127.9.194.253@KRBTEST.COM (short username kudu)
I20260807 22:53:12.561339 24465 webserver.cc:460] Webserver started at http://127.9.194.253:33777/ using document root <none> and password file <none>
I20260807 22:53:12.562026 24465 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:12.562167 24465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:12.562511 24465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:12.565521 24465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data/instance:
uuid: "1b0c7da37cc7406db4cd218935ca473f"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "8762a9dc318f6e1eed3a94747ae543bb"
server_key_iv: "203b5db40d2b5b1a3cb35ff00bc6454d"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.566303 24465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal/instance:
uuid: "1b0c7da37cc7406db4cd218935ca473f"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "8762a9dc318f6e1eed3a94747ae543bb"
server_key_iv: "203b5db40d2b5b1a3cb35ff00bc6454d"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.571377 24465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:53:12.574815 24480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:12.576292 24465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:12.576470 24465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "1b0c7da37cc7406db4cd218935ca473f"
format_stamp: "Formatted at 2026-08-07 22:53:12 on dist-test-slave-5k54"
server_key: "8762a9dc318f6e1eed3a94747ae543bb"
server_key_iv: "203b5db40d2b5b1a3cb35ff00bc6454d"
server_key_version: "encryptionkey@0"
I20260807 22:53:12.576660 24465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:12.600893 24465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:12.604861 24465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:12.605151 24465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:12.623670 24465 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.253:33391
I20260807 22:53:12.623695 24533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.253:33391 every 8 connection(s)
I20260807 22:53:12.625072 24465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
I20260807 22:53:12.628695 24535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:12.629323 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24465
I20260807 22:53:12.629523 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-1/wal/instance
I20260807 22:53:12.630005 9995 external_mini_cluster.cc:1428] Setting key ad4883f61ba54434c710be5e50cf6991
I20260807 22:53:12.635149 24535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.194.252@KRBTEST.COM; defaulting to no policy
I20260807 22:53:12.658769 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.638058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.9.194.254:38821)
0807 22:53:12.638259 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:12.638266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:12.639680 (+ 1414us) server_negotiation.cc:407] Connection header received
0807 22:53:12.642504 (+ 2824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:12.642508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:12.642575 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:12.642666 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:12.644806 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.645547 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.648157 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.648580 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.648675 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.652930 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:12.652934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:12.652938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:12.652971 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:12.655783 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.656522 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.656532 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.656535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.656651 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.657082 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.657087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.657103 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.657380 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:12.657543 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:12.658221 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 22:53:12.658452 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.252@KRBTEST.COM" created.
I20260807 22:53:12.659922 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.638651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38821 (local address 127.0.0.1:42178)
0807 22:53:12.639495 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.639556 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:12.642122 (+ 2566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:12.642923 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:12.642936 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:12.643834 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:12.644600 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.644628 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.645729 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.645738 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.647994 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.648013 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.648806 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.648815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.649144 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.649917 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:12.649949 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:12.652689 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:12.655933 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.655942 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.655957 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.656390 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.656762 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.656767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.656771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.656960 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.657516 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:12.657524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:12.657853 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 22:53:12.658117 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 22:53:12.658607 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":336,"threads_started":1}
I20260807 22:53:12.661774 24535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:53:12.679636 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.662543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54420)
0807 22:53:12.662710 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.662729 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:12.662833 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:12.664153 (+ 1320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:12.664156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:12.664511 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:12.664652 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.664660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.666649 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.666653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.667665 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.667678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.669073 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.669078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.669264 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.669900 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:12.669926 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:12.672511 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:12.676998 (+ 4487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.677002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.677005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.677414 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.677841 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.677845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.677847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.678003 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.678790 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:12.678793 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:12.678913 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 22:53:12.679027 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:53:12.679230 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":3}
I20260807 22:53:12.680258 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.662844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.9.194.253:33391)
0807 22:53:12.663611 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:12.663622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:12.663660 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:53:12.663719 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:12.663729 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:12.663978 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:12.664136 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:12.664822 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.666512 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.667818 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.668898 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.669058 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.672737 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:12.672754 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:12.672774 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:12.672818 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:12.676850 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.677555 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.677567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.677573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.677722 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.678115 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.678122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.678128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.678656 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:12.678879 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:12.679213 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 22:53:12.679556 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":538,"thread_start_us":383,"threads_started":1}
I20260807 22:53:12.683285 24535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:53:12.684664 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.684185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41214)
0807 22:53:12.684403 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.684528 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 22:53:12.684952 24483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.686539 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.686196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41216)
0807 22:53:12.686359 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.686444 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:53:12.686740 24535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
I20260807 22:53:12.696309 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:37369
--webserver_interface=127.9.194.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:12.736042 24535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 1
I20260807 22:53:12.737192 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.736724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41222)
0807 22:53:12.736926 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.737040 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 22:53:12.738168 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.737822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41234)
0807 22:53:12.737993 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.738056 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 22:53:12.738373 24535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.806572 24535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 2
I20260807 22:53:12.807721 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.807223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41248)
0807 22:53:12.807449 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.807563 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 22:53:12.808727 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.808379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41260)
0807 22:53:12.808547 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.808613 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:53:12.808946 24535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.826761 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } attempt: 5
Aug 07 22:53:12 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:53:12.846287 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.827722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.9.194.253:33391)
0807 22:53:12.827931 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:12.827937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:12.827959 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:53:12.828102 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:12.828106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:12.828168 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:12.828276 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:12.829086 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.829782 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.830937 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.831481 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.831625 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.840725 (+ 9100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:12.840734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:12.840742 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:12.840795 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:12.843542 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.844364 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.844367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.844370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.844487 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:12.844908 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:12.844911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:12.844913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:12.845189 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:12.845299 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:12.845758 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:53:12.846015 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:53:12.846329 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.827600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54426)
0807 22:53:12.827793 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.827818 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:12.827950 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:12.828316 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:12.828321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:12.828738 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:12.828928 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.828937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.829923 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.829931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.830790 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:12.830801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:12.831639 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:12.831644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:12.831846 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:12.837996 (+ 6150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:12.838016 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:12.840464 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:12.843701 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.843707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.843711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.844240 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.844599 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:12.844604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:12.844608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:12.844803 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:12.845310 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:12.845315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:12.845506 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 22:53:12.845670 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 22:53:12.845986 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 22:53:12.847514 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:53:12.848754 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.848257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41266)
0807 22:53:12.848461 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.848555 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 22:53:12.849854 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.849396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41272)
0807 22:53:12.849582 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.849706 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:53:12.850099 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.900172 24535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 3
I20260807 22:53:12.901304 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.900840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41282)
0807 22:53:12.901049 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.901149 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 22:53:12.901364 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 1
I20260807 22:53:12.902490 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.901991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41286)
0807 22:53:12.902161 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.902299 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:53:12.902652 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.902122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41294)
0807 22:53:12.902390 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.902501 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
W20260807 22:53:12.902906 24535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.903604 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.903202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41308)
0807 22:53:12.903358 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.903486 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 22:53:12.903800 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:12.972062 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 2
I20260807 22:53:12.973340 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.972849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41312)
0807 22:53:12.973069 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.973185 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:53:12.974380 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:12.974078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41316)
0807 22:53:12.974205 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:12.974285 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:53:12.974624 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
W20260807 22:53:13.035693 24542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:13.035997 24542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:13.036211 24542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:13.043228 24542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:13.043322 24542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:13.043447 24542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:13.043562 24542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:13.043643 24542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:13.046139 24535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 4
I20260807 22:53:13.047485 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.046824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41326)
0807 22:53:13.047123 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.047236 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 22:53:13.048429 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.048110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41334)
0807 22:53:13.048253 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.048330 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:53:13.048646 24535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:13.054061 24542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38821,127.9.194.253:33391,127.9.194.252:37369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:37369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.194.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:13.055480 24542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:13.056941 24542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:13.068782 24547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:13.068827 24548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:13.068717 24550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:13.074950 24542 server_base.cc:1034] running on GCE node
I20260807 22:53:13.075927 24542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:13.077334 24542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:13.078536 24542 hybrid_clock.cc:648] HybridClock initialized: now 1786143193078467 us; error 55 us; skew 500 ppm
I20260807 22:53:13.078886 24542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:53:13.080874 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 3
I20260807 22:53:13.081911 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.081462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41348)
0807 22:53:13.081687 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.081765 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143193, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:13.082508 24542 init.cc:377] Logged in from keytab as kudu/127.9.194.252@KRBTEST.COM (short username kudu)
I20260807 22:53:13.082898 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.082523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41358)
0807 22:53:13.082735 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.082798 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 22:53:13.083256 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:37369: Network error: Client connection negotiation failed: client connection to 127.9.194.252:37369: connect: Connection refused (error 111)
I20260807 22:53:13.084266 24542 webserver.cc:460] Webserver started at http://127.9.194.252:36677/ using document root <none> and password file <none>
I20260807 22:53:13.084998 24542 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:13.085132 24542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:13.085450 24542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:13.088500 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data/instance:
uuid: "7adbc92e5f50493cb9eb0857da6c12d8"
format_stamp: "Formatted at 2026-08-07 22:53:13 on dist-test-slave-5k54"
server_key: "43f9863124a265716e05aa13c09556bf"
server_key_iv: "e51f6a29a7a4f28f9e0374567724b044"
server_key_version: "encryptionkey@0"
I20260807 22:53:13.089336 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal/instance:
uuid: "7adbc92e5f50493cb9eb0857da6c12d8"
format_stamp: "Formatted at 2026-08-07 22:53:13 on dist-test-slave-5k54"
server_key: "43f9863124a265716e05aa13c09556bf"
server_key_iv: "e51f6a29a7a4f28f9e0374567724b044"
server_key_version: "encryptionkey@0"
I20260807 22:53:13.094425 24542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:53:13.097868 24557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:13.099193 24542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:53:13.099398 24542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "7adbc92e5f50493cb9eb0857da6c12d8"
format_stamp: "Formatted at 2026-08-07 22:53:13 on dist-test-slave-5k54"
server_key: "43f9863124a265716e05aa13c09556bf"
server_key_iv: "e51f6a29a7a4f28f9e0374567724b044"
server_key_version: "encryptionkey@0"
I20260807 22:53:13.099661 24542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:13.114759 24542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:13.118842 24542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:13.119129 24542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:13.138226 24542 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.252:37369
I20260807 22:53:13.138264 24613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.252:37369 every 8 connection(s)
I20260807 22:53:13.139665 24542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
I20260807 22:53:13.142231 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24542
I20260807 22:53:13.142455 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142983724342-9995-0/minicluster-data/master-2/wal/instance
I20260807 22:53:13.143064 9995 external_mini_cluster.cc:1428] Setting key 69d3ac1b0e884f5b442f8039eabf7c95
I20260807 22:53:13.143862 24614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:13.153662 24614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143193, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:13.167114 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.145015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.9.194.252:37369)
0807 22:53:13.145771 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.145782 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.145822 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:53:13.147343 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.147363 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.147816 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.148143 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.149984 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.151290 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.153169 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.153814 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.153950 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.159091 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.159103 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.159121 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.159153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.162828 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.163591 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.163598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.163603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.163789 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.164223 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.164235 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.164237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.164872 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.165177 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.165640 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.166100 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":380,"threads_started":1}
I20260807 22:53:13.172529 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.156762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:38821 (local address 127.0.0.1:42184)
0807 22:53:13.157541 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.157580 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:13.157734 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:13.158128 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:13.158133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:13.158666 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:13.158968 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.158979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.160170 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.160177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.161981 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.161995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.162792 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.162799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.162998 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.163626 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:13.163645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:13.166100 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:13.169591 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.169599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.169604 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.170223 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.170830 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.170836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.170842 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.170993 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.171694 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:13.171700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:13.171860 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 22:53:13.172017 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:53:13.172259 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":415,"threads_started":1}
I20260807 22:53:13.173077 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.156904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.9.194.254:38821)
0807 22:53:13.157166 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.157174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.157690 (+ 516us) server_negotiation.cc:407] Connection header received
0807 22:53:13.157861 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.157867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.157946 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.158061 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.159139 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.159969 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.162140 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.162599 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.162696 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.166287 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.166292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.166296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.166328 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.169359 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.170379 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.170401 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.170403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.170646 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.171130 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.171135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.171138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.171564 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.171736 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.172213 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.172429 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:53:13.174402 24614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143193, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:53:13.187503 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.170245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.9.194.254:38821)
0807 22:53:13.170879 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.170885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.170907 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:53:13.170950 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.170955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.171021 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.171121 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.172825 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.173855 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.175378 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.175990 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.176090 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.181363 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.181367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.181370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.181398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.184704 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.185481 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.185487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.185492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.185635 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.185986 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.185992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.185995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.186349 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.186629 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.186817 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.187066 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":294,"threads_started":1}
I20260807 22:53:13.189869 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.175001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:33391 (local address 127.0.0.1:54438)
0807 22:53:13.175198 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.175216 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:13.175315 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:13.175701 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:13.175705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:13.176023 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:13.176218 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.176231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.177573 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.177580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.179059 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.179075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.180158 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.180164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.180458 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.181046 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:13.181072 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:13.183065 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:13.186990 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.186997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.187002 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.187417 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.187897 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.187902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.187906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.188050 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.188817 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:13.188820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:13.189019 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 22:53:13.189185 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 22:53:13.189511 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 22:53:13.190845 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.175167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.9.194.253:33391)
0807 22:53:13.175372 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.175378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.175396 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:53:13.175505 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.175510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.175580 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.175683 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.176470 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.177418 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.179216 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.179946 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.180090 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.183254 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.183261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.183267 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.183314 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.186825 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.187582 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.187587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.187590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.187741 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.188162 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.188169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.188172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.188461 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.188610 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.190232 (+ 1622us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.190479 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:53:13.191781 24614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143193, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:53:13.204799 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.192514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.9.194.252:37369)
0807 22:53:13.192658 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.192662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.192849 (+ 187us) server_negotiation.cc:407] Connection header received
0807 22:53:13.193029 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.193034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.193092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.193183 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.193824 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.194493 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.195596 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.196014 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.196145 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.199026 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.199035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.199041 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.199085 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.201821 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.202570 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.202573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.202576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.202734 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.203140 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.203146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.203151 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.203551 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.203701 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.204179 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.204511 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":48}
I20260807 22:53:13.204803 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.192665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41376)
0807 22:53:13.192767 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.192783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:13.192904 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:13.193200 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:13.193204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:13.193568 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:13.193688 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.193708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.194611 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.194618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.195411 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.195470 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.196160 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.196165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.196312 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.196845 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:13.196860 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:13.198832 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:13.201954 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.201961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.201966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.202443 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.202851 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.202857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.202862 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.203031 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.203663 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:13.203667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:13.203813 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 22:53:13.204049 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 22:53:13.204312 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:53:13.206506 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.189102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.9.194.253:33391)
0807 22:53:13.189694 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.189701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.189728 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:53:13.189774 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.189779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.189855 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.189956 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.191947 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.193064 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.194304 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.194732 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.194883 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.199903 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.199912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.199918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.199960 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.202955 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.203825 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.203830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.203833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.203979 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.204424 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.204429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.204432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.204768 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.205043 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.205891 (+ 848us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.206130 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":267,"threads_started":1}
I20260807 22:53:13.210580 24614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: Bootstrap starting.
I20260807 22:53:13.211413 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:53:13.214001 24614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:13.215059 24614 log.cc:824] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:13.219086 24614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: No bootstrap required, opened a new log
I20260807 22:53:13.233284 24614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.233763 24614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:13.233901 24614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adbc92e5f50493cb9eb0857da6c12d8, State: Initialized, Role: FOLLOWER
I20260807 22:53:13.234781 24614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.238479 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.217161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.9.194.254:38821)
0807 22:53:13.217347 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.217353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.217488 (+ 135us) server_negotiation.cc:407] Connection header received
0807 22:53:13.217767 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.217773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.217875 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.217960 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.221108 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.222519 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.225559 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.226475 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.226599 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.231142 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.231152 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.231158 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.231207 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.235019 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.236042 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.236050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.236055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.236212 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.236676 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.236682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.236684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.237041 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.237269 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.237869 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.238111 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:53:13.241776 24630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.242240 24630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.244159 24614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:13.249516 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.217707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.9.194.252:37369)
0807 22:53:13.217899 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.217907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.231650 (+ 13743us) server_negotiation.cc:407] Connection header received
0807 22:53:13.231916 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.231926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.232025 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.232097 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.234002 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.235104 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.237159 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.237825 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.237913 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.241966 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.241975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.241980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.242022 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.245810 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.246805 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.246813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.246817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.246977 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.247401 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.247406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.247409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.247792 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.248766 (+ 974us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.248860 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.249150 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:53:13.249855 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.216625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54458 (local address 127.9.194.253:33391)
0807 22:53:13.216833 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.216838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.228310 (+ 11472us) server_negotiation.cc:407] Connection header received
0807 22:53:13.228833 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.228839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.228933 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.229023 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.232366 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.233370 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.236116 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.237003 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.237129 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.242799 (+ 5670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.242807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.242813 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.242863 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.246460 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.247541 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.247548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.247553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.247686 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.248187 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.248193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.248196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.248562 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.248803 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.249350 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.249580 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 22:53:13.259773 24641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:53:13.259910 24641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:53:13.260483 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 4
I20260807 22:53:13.257107 24542 master_runner.cc:186] Error getting master registration for 127.9.194.254:38821: OK, instance_id { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" instance_seqno: 1786143192112431 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:53:13.266690 24542 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
I20260807 22:53:13.278192 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.261627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.9.194.252:37369)
0807 22:53:13.261797 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.261803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.261820 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:53:13.261872 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.261876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.261984 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.262090 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.263148 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.264456 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.266317 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.267041 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.267172 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.270872 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.270878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.270883 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.270918 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.274522 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.275267 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.275272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.275276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.276221 (+ 945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.276743 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.276750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.276754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.277056 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.277218 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.277591 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.277799 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:53:13.278189 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.261000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41394)
0807 22:53:13.261321 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.261340 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:13.261442 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:13.262258 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:13.262263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:13.262770 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:13.262980 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.262993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.264615 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.264621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.266154 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.266169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.267195 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.267199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.267346 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.268054 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:13.268072 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:13.270688 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:13.274668 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.274672 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.274675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.275136 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.276430 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.276438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.276445 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.276626 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.277192 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:13.277197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:13.277355 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 22:53:13.277501 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 22:53:13.277783 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260807 22:53:13.283404 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: Bootstrap starting.
I20260807 22:53:13.286847 24457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:13.287969 24457 log.cc:824] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:13.290777 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: No bootstrap required, opened a new log
I20260807 22:53:13.298563 24457 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.298856 24457 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:13.299014 24457 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b953fb3773486c8231b405c23aef4c, State: Initialized, Role: FOLLOWER
I20260807 22:53:13.299676 24457 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.300597 24644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.301020 24644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.301772 24457 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:53:13.312341 24656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:53:13.312474 24656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:53:13.312747 24385 master_runner.cc:186] Error getting master registration for 127.9.194.253:33391: OK, instance_id { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" instance_seqno: 1786143192618347 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:53:13.324925 24385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:53:13.347868 24535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } attempt: 5
W20260807 22:53:13.350576 24407 tablet.cc:2369] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:53:13 dist-test-slave-5k54 krb5kdc[24372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
I20260807 22:53:13.366943 24630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:13.367280 24630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.368984 24630 leader_election.cc:288] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3b953fb3773486c8231b405c23aef4c (127.9.194.254:38821), 1b0c7da37cc7406db4cd218935ca473f (127.9.194.253:33391)
I20260807 22:53:13.369987 24431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b953fb3773486c8231b405c23aef4c" is_pre_election: true
I20260807 22:53:13.370555 24431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7adbc92e5f50493cb9eb0857da6c12d8 in term 0.
W20260807 22:53:13.371315 24564 tablet.cc:2369] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:13.371660 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.348544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:37369 (local address 127.0.0.1:41400)
0807 22:53:13.349015 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:13.349034 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:13.349176 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:13.349620 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:13.349626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:13.350252 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:13.350517 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.350534 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.353467 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.353474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.354929 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:13.354946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.356136 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.356143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:13.356363 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.357112 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:13.357136 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:13.360197 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:13.364509 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.364516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.364520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.365214 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.365691 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:13.365697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:13.365701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:13.365894 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:13.366641 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:13.366648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:13.366839 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 22:53:13.367009 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 22:53:13.367326 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345}
I20260807 22:53:13.371904 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:13.348660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.9.194.252:37369)
0807 22:53:13.348862 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:13.348869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:13.349309 (+ 440us) server_negotiation.cc:407] Connection header received
0807 22:53:13.349377 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:13.349383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:13.349457 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:13.349554 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:13.350711 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.351987 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.355108 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:13.355962 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:13.356125 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:13.360393 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:13.360399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:13.360405 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:13.360443 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:13.364352 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.365347 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.365353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.365357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.365550 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:13.366025 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:13.366031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:13.366036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:13.366424 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:13.366567 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:13.370625 (+ 4058us) server_negotiation.cc:299] Negotiation successful
0807 22:53:13.370924 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:53:13.373081 24562 leader_election.cc:302] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7adbc92e5f50493cb9eb0857da6c12d8, d3b953fb3773486c8231b405c23aef4c; no voters:
I20260807 22:53:13.374317 24630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:53:13.374468 24630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:53:13.374567 24630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:13.377892 24630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.378194 24535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: Bootstrap starting.
I20260807 22:53:13.379137 24431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b953fb3773486c8231b405c23aef4c"
I20260807 22:53:13.379356 24431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:13.382823 24431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7adbc92e5f50493cb9eb0857da6c12d8 in term 1.
I20260807 22:53:13.382987 24535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:13.383544 24562 leader_election.cc:302] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7adbc92e5f50493cb9eb0857da6c12d8, d3b953fb3773486c8231b405c23aef4c; no voters:
I20260807 22:53:13.384456 24535 log.cc:824] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:13.387594 24630 leader_election.cc:288] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [CANDIDATE]: Term 1 election: Requested vote from peers d3b953fb3773486c8231b405c23aef4c (127.9.194.254:38821), 1b0c7da37cc7406db4cd218935ca473f (127.9.194.253:33391)
I20260807 22:53:13.388355 24630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:13.388967 24630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [term 1 LEADER]: Becoming Leader. State: Replica: 7adbc92e5f50493cb9eb0857da6c12d8, State: Running, Role: LEADER
I20260807 22:53:13.390923 24630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.394958 24535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: No bootstrap required, opened a new log
I20260807 22:53:13.409114 24535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.409543 24535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:13.409667 24535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0c7da37cc7406db4cd218935ca473f, State: Initialized, Role: FOLLOWER
I20260807 22:53:13.410501 24535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } }
I20260807 22:53:13.410669 24658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adbc92e5f50493cb9eb0857da6c12d8. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.411567 24658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:13.417560 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.417989 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.418661 24662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:13.419951 24535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:13.420024 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b0c7da37cc7406db4cd218935ca473f"
I20260807 22:53:13.420603 24506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:13.421983 24662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:13.424327 24506 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7adbc92e5f50493cb9eb0857da6c12d8 in term 1.
I20260807 22:53:13.420071 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b0c7da37cc7406db4cd218935ca473f" is_pre_election: true
I20260807 22:53:13.424912 24507 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7adbc92e5f50493cb9eb0857da6c12d8 in term 1. Re-sending same reply.
I20260807 22:53:13.442907 24507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [term 1 FOLLOWER]: Refusing update from remote peer 7adbc92e5f50493cb9eb0857da6c12d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:13.443579 24431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [term 1 FOLLOWER]: Refusing update from remote peer 7adbc92e5f50493cb9eb0857da6c12d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:13.445138 24630 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:13.445871 24630 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:13.473245 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adbc92e5f50493cb9eb0857da6c12d8. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.473724 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.476742 24664 mvcc.cc:205] Tried to move back new op lower bound from 7316042520327942144 to 7316042520151007232. Current Snapshot: MvccSnapshot[applied={T|T < 7316042520327942144}]
I20260807 22:53:13.484046 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.484350 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.493052 24682 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:53:13.493045 24630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.493413 24630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:13.493791 24671 mvcc.cc:205] Tried to move back new op lower bound from 7316042520327942144 to 7316042520151007232. Current Snapshot: MvccSnapshot[applied={T|T < 7316042520327942144 or (T in {7316042520327942144})}]
I20260807 22:53:13.494663 24662 catalog_manager.cc:1347] Generated new cluster ID: d7c6910c45c6436eb9e7e3813ce11f24
I20260807 22:53:13.494760 24662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:13.500133 24644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adbc92e5f50493cb9eb0857da6c12d8. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.500459 24644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.514384 24658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.514705 24658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:13.516031 24644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b953fb3773486c8231b405c23aef4c" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38821 } } peers { permanent_uuid: "1b0c7da37cc7406db4cd218935ca473f" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 33391 } } peers { permanent_uuid: "7adbc92e5f50493cb9eb0857da6c12d8" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 37369 } } }
I20260807 22:53:13.516312 24644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:53:13.532027 24672 mvcc.cc:205] Tried to move back new op lower bound from 7316042520327942144 to 7316042520151007232. Current Snapshot: MvccSnapshot[applied={T|T < 7316042520327942144 or (T in {7316042520327942144})}]
I20260807 22:53:13.538537 24662 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:53:13.540509 24662 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:13.553586 24662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7adbc92e5f50493cb9eb0857da6c12d8: Generated new TSK 0
I20260807 22:53:13.554772 24662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:53:14.314627 24656 catalog_manager.cc:1259] Loaded cluster ID: d7c6910c45c6436eb9e7e3813ce11f24
I20260807 22:53:14.314733 24656 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: loading cluster ID for follower catalog manager: success
I20260807 22:53:14.317181 24656 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: acquiring CA information for follower catalog manager: success
I20260807 22:53:14.318905 24656 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3b953fb3773486c8231b405c23aef4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:53:14.497231 24682 catalog_manager.cc:1259] Loaded cluster ID: d7c6910c45c6436eb9e7e3813ce11f24
I20260807 22:53:14.497382 24682 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: loading cluster ID for follower catalog manager: success
I20260807 22:53:14.499997 24682 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: acquiring CA information for follower catalog manager: success
I20260807 22:53:14.502056 24682 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b0c7da37cc7406db4cd218935ca473f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:53:14.646800 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24385
I20260807 22:53:14.648092 24452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:14.805509 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24385
I20260807 22:53:14.831593 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24465
I20260807 22:53:14.832978 24529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:14.953910 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24465
I20260807 22:53:14.976644 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24542
I20260807 22:53:14.977768 24609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:15.087083 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24542
2026-08-07T22:53:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4848 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:53:15 dist-test-slave-5k54 krb5kdc[24697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:53:15 dist-test-slave-5k54 krb5kdc[24697](info): set up 2 sockets
Aug 07 22:53:15 dist-test-slave-5k54 krb5kdc[24697](info): commencing operation
krb5kdc: starting...
W20260807 22:53:16.236194 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:53:16 dist-test-slave-5k54 krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:53:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:53:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:53:16.504586 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44225
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:44225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:16.846920 24711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:16.847296 24711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:16.847506 24711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:16.855044 24711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:16.855170 24711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:16.855266 24711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:16.855365 24711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:16.855448 24711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:16.865644 24711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:44225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:16.867017 24711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:16.868459 24711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:16.882863 24716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:16.882812 24719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:16.882778 24717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:16.884943 24711 server_base.cc:1034] running on GCE node
I20260807 22:53:16.885761 24711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:16.887184 24711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:16.888415 24711 hybrid_clock.cc:648] HybridClock initialized: now 1786143196888347 us; error 76 us; skew 500 ppm
I20260807 22:53:16.888780 24711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:16 dist-test-slave-5k54 krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143196, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:16.892028 24711 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:16.893469 24711 webserver.cc:460] Webserver started at http://127.9.194.254:43879/ using document root <none> and password file <none>
I20260807 22:53:16.894115 24711 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:16.894212 24711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:16.894459 24711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:16.897398 24711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "971f762e0d12428685c28de23a6b2fab"
format_stamp: "Formatted at 2026-08-07 22:53:16 on dist-test-slave-5k54"
server_key: "8f74886825f50c200a9e4a2fc957e69c"
server_key_iv: "0e450d511bb5d02581635502869647e6"
server_key_version: "encryptionkey@0"
I20260807 22:53:16.898098 24711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "971f762e0d12428685c28de23a6b2fab"
format_stamp: "Formatted at 2026-08-07 22:53:16 on dist-test-slave-5k54"
server_key: "8f74886825f50c200a9e4a2fc957e69c"
server_key_iv: "0e450d511bb5d02581635502869647e6"
server_key_version: "encryptionkey@0"
I20260807 22:53:16.903105 24711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 22:53:16.906409 24726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:16.907722 24711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:16.907893 24711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "971f762e0d12428685c28de23a6b2fab"
format_stamp: "Formatted at 2026-08-07 22:53:16 on dist-test-slave-5k54"
server_key: "8f74886825f50c200a9e4a2fc957e69c"
server_key_iv: "0e450d511bb5d02581635502869647e6"
server_key_version: "encryptionkey@0"
I20260807 22:53:16.908075 24711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:16.940902 24711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:16.945225 24711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:16.945473 24711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:16.963934 24711 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:44225
I20260807 22:53:16.963971 24777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:44225 every 8 connection(s)
I20260807 22:53:16.965277 24711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:16.968993 24778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:16.973587 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24711
I20260807 22:53:16.973816 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:53:16.974786 9995 external_mini_cluster.cc:1428] Setting key a55ea2420fdf260a20b46005e37dccb6
I20260807 22:53:16.980902 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab: Bootstrap starting.
I20260807 22:53:16.984596 24778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:16.985787 24778 log.cc:824] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:16.988729 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab: No bootstrap required, opened a new log
Aug 07 22:53:16 dist-test-slave-5k54 krb5kdc[24697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:16.997918 24778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } }
I20260807 22:53:16.998293 24778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:16.998421 24778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 971f762e0d12428685c28de23a6b2fab, State: Initialized, Role: FOLLOWER
I20260807 22:53:16.998687 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:16.976709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.9.194.254:44225)
0807 22:53:16.977547 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:16.977561 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:16.977604 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:53:16.979478 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:16.979509 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:16.979878 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:16.980189 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:16.982035 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:16.983350 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:16.985309 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:16.986320 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:16.986557 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:16.991131 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:16.991138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:16.991152 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:16.991194 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:16.994445 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:16.995258 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:16.995263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:16.995269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:16.995494 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:16.995916 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:16.995919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:16.995921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:16.996442 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:16.996658 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:16.997119 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 22:53:16.997506 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":407,"threads_started":1}
I20260807 22:53:16.999073 24778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } }
I20260807 22:53:16.999332 24778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:16.999503 24778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:16.999687 24778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:17.002805 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:53:17.002780 24778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } }
I20260807 22:53:17.003275 24778 leader_election.cc:302] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 971f762e0d12428685c28de23a6b2fab; no voters:
I20260807 22:53:17.003906 24778 leader_election.cc:288] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:53:17.004094 24784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:17.005883 24778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:17.006556 24784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [term 1 LEADER]: Becoming Leader. State: Replica: 971f762e0d12428685c28de23a6b2fab, State: Running, Role: LEADER
I20260807 22:53:17.007126 24784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } }
I20260807 22:53:17.010517 24785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "971f762e0d12428685c28de23a6b2fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } } }
I20260807 22:53:17.011009 24785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:17.011571 24787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 971f762e0d12428685c28de23a6b2fab. Latest consensus state: current_term: 1 leader_uuid: "971f762e0d12428685c28de23a6b2fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971f762e0d12428685c28de23a6b2fab" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44225 } } }
I20260807 22:53:17.011906 24787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:17.016417 24799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:17.020599 24799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:17.028717 24799 catalog_manager.cc:1347] Generated new cluster ID: 803ed09fe8874ece9c5d2826c3d5c91b
I20260807 22:53:17.028841 24799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:17.043061 24799 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:53:17.044929 24799 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:17.063058 24799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 971f762e0d12428685c28de23a6b2fab: Generated new TSK 0
I20260807 22:53:17.064136 24799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:53:17.116374 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24711
I20260807 22:53:17.117548 24773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:17.213791 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24711
2026-08-07T22:53:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2124 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:53:17 dist-test-slave-5k54 krb5kdc[24808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:53:17 dist-test-slave-5k54 krb5kdc[24808](info): set up 2 sockets
Aug 07 22:53:17 dist-test-slave-5k54 krb5kdc[24808](info): commencing operation
krb5kdc: starting...
W20260807 22:53:18.368693 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:53:18 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:53:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:53:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:53:18.647171 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:18.990029 24821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:18.990322 24821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:18.990412 24821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:18.997565 24821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:18.997668 24821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:18.997714 24821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:18.997766 24821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:18.997802 24821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:19.007352 24821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:19.008693 24821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:19.009970 24821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:19.021452 24827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:19.021488 24829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:19.021538 24826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:19.024891 24821 server_base.cc:1034] running on GCE node
I20260807 22:53:19.025710 24821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:19.027158 24821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:19.028383 24821 hybrid_clock.cc:648] HybridClock initialized: now 1786143199028341 us; error 50 us; skew 500 ppm
I20260807 22:53:19.028721 24821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:19 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143199, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:19.037583 24821 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:19.039072 24821 webserver.cc:460] Webserver started at http://127.9.194.254:43195/ using document root <none> and password file <none>
I20260807 22:53:19.039847 24821 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:19.039997 24821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:19.040304 24821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:19.043367 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "8a2e3df33c004a58af245c9db8bd74f7"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "95d5ebc820b038844dce0fa291dcee25"
server_key_iv: "28ed5c511fee9baa2163a1669aa6526b"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.044203 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "8a2e3df33c004a58af245c9db8bd74f7"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "95d5ebc820b038844dce0fa291dcee25"
server_key_iv: "28ed5c511fee9baa2163a1669aa6526b"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.049312 24821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:53:19.052754 24837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:19.054136 24821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:53:19.054322 24821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a2e3df33c004a58af245c9db8bd74f7"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "95d5ebc820b038844dce0fa291dcee25"
server_key_iv: "28ed5c511fee9baa2163a1669aa6526b"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.054520 24821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:19.068406 24821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:19.072841 24821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:19.073158 24821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:19.092523 24821 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:36625
I20260807 22:53:19.092561 24888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:36625 every 8 connection(s)
I20260807 22:53:19.093981 24821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:19.097944 24889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:19.103689 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24821
I20260807 22:53:19.103926 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:53:19.104606 9995 external_mini_cluster.cc:1428] Setting key bfffc1e20a9a12ae67e42588bbf6c40f
I20260807 22:53:19.109717 24889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap starting.
I20260807 22:53:19.112979 24889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:19.114074 24889 log.cc:824] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:19.117056 24889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: No bootstrap required, opened a new log
Aug 07 22:53:19 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:19.125183 24889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:19.125588 24889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:19.125701 24889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Initialized, Role: FOLLOWER
I20260807 22:53:19.126363 24889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:19.126624 24889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:19.126811 24889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:19.127007 24889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:19.128265 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:19.106802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.9.194.254:36625)
0807 22:53:19.107724 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:19.107739 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:19.107779 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:53:19.109819 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:19.109865 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:19.110347 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:19.110745 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:19.112393 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.113757 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.115646 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.116318 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.116528 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:19.120532 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:19.120540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:19.120555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:19.120595 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:19.123915 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:19.124643 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:19.124650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:19.124656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:19.124834 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:19.125232 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:19.125236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:19.125238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:19.125884 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:19.126250 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:19.126642 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 22:53:19.127148 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":403,"threads_started":1}
I20260807 22:53:19.129954 24889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:19.130473 24889 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2e3df33c004a58af245c9db8bd74f7; no voters:
I20260807 22:53:19.131282 24889 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:53:19.131417 24895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:19.132241 24895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Running, Role: LEADER
I20260807 22:53:19.132853 24895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:19.133749 24889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:19.135898 24897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:19.135982 24896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2e3df33c004a58af245c9db8bd74f7. Latest consensus state: current_term: 1 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:19.136307 24897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:19.137928 24896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:19.138043 24901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:19.142521 24901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:19.152175 24901 catalog_manager.cc:1347] Generated new cluster ID: e75b7e716980498f8607449016e65722
I20260807 22:53:19.152307 24901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:19.172152 24901 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:53:19.174014 24901 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:19.185823 24901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Generated new TSK 0
I20260807 22:53:19.186924 24901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:53:19.257462 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:19.593956 24917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:19.594236 24917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:19.594398 24917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:19.601552 24917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:19.601660 24917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:19.601835 24917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:19.611546 24917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:19.612895 24917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:19.614271 24917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:19.627507 24927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:19.627542 24925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:19.632082 24924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:19.633697 24917 server_base.cc:1034] running on GCE node
I20260807 22:53:19.634426 24917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:19.635566 24917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:19.636776 24917 hybrid_clock.cc:648] HybridClock initialized: now 1786143199636733 us; error 36 us; skew 500 ppm
I20260807 22:53:19.637113 24917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:19 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143199, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:19.640657 24917 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:19.642498 24917 webserver.cc:460] Webserver started at http://127.9.194.193:40617/ using document root <none> and password file <none>
I20260807 22:53:19.643271 24917 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:19.643414 24917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:19.643765 24917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:19.646814 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "d7d1d78776a74301b8cf4757700f4bde"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "8c94db0018e11adfff70fc51fef374d2"
server_key_iv: "37ee91ff5063d11b8c3b172481dfbf07"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.647696 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "d7d1d78776a74301b8cf4757700f4bde"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "8c94db0018e11adfff70fc51fef374d2"
server_key_iv: "37ee91ff5063d11b8c3b172481dfbf07"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.652927 24917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:53:19.656656 24934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:19.658075 24917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:53:19.658242 24917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "d7d1d78776a74301b8cf4757700f4bde"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "8c94db0018e11adfff70fc51fef374d2"
server_key_iv: "37ee91ff5063d11b8c3b172481dfbf07"
server_key_version: "encryptionkey@0"
I20260807 22:53:19.658460 24917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:19.700850 24917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:19.705353 24917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:19.705693 24917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:19.706763 24917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:19.708146 24917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:19.708221 24917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:19.708304 24917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:19.708348 24917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:19.742231 24917 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:40459
I20260807 22:53:19.742277 25050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:40459 every 8 connection(s)
I20260807 22:53:19.743556 24917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:19.746057 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 24917
I20260807 22:53:19.746268 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:53:19.747139 9995 external_mini_cluster.cc:1428] Setting key a6bef12a32cb30f5d55ad67bd4d95ef8
Aug 07 22:53:19 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143199, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:19.765777 25054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:19.746574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:41251 (local address 127.9.194.254:36625)
0807 22:53:19.747106 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:19.747112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:19.748151 (+ 1039us) server_negotiation.cc:407] Connection header received
0807 22:53:19.750532 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:19.750536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:19.750601 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:19.750684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:19.752847 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.753603 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.755174 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.755638 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.755777 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:19.759461 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:19.759471 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:19.759476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:19.759515 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:19.762605 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:19.763462 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:19.763467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:19.763471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:19.763596 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:19.764035 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:19.764041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:19.764044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:19.764306 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:19.764457 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:19.765177 (+ 720us) server_negotiation.cc:299] Negotiation successful
0807 22:53:19.765409 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":254,"threads_started":1}
I20260807 22:53:19.766785 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:19.747114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:41251)
0807 22:53:19.747965 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:19.748011 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:19.750226 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:19.750972 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:19.750987 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:19.751825 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:19.752661 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:19.752688 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.753753 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.753759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:19.755018 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:19.755030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:19.755864 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:19.755873 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:19.756117 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:19.756787 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:19.756809 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:19.759208 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:19.762738 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:19.762747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:19.762764 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:19.763323 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:19.763707 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:19.763712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:19.763716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:19.763917 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:19.764441 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:19.764449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:19.764806 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 22:53:19.765077 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 22:53:19.765632 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":337,"threads_started":1}
I20260807 22:53:19.768388 25051 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:19.768721 25051 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:19.769435 25051 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:19.771603 24854 ts_manager.cc:194] Registered new tserver with Master: d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459)
I20260807 22:53:19.773079 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:41251
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:53:19.810407 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:20.148335 25058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:20.148597 25058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:20.148687 25058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:20.155660 25058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:20.155772 25058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:20.155984 25058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:20.165822 25058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:20.167084 25058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:20.168408 25058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:20.181444 25065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:20.181483 25067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:20.181506 25064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:20.182456 25058 server_base.cc:1034] running on GCE node
I20260807 22:53:20.183048 25058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:20.184049 25058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:20.185271 25058 hybrid_clock.cc:648] HybridClock initialized: now 1786143200185233 us; error 47 us; skew 500 ppm
I20260807 22:53:20.185580 25058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:20 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:20.188714 25058 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:20.190078 25058 webserver.cc:460] Webserver started at http://127.9.194.194:44803/ using document root <none> and password file <none>
I20260807 22:53:20.190743 25058 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:20.190882 25058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:20.191183 25058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:20.194064 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "52e3d8fb84b54488811f474e039e10a6"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "da804508ec8c2609828411914968db7a"
server_key_iv: "e996226deeb38f1a4a8273008f51c89a"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.194823 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "52e3d8fb84b54488811f474e039e10a6"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "da804508ec8c2609828411914968db7a"
server_key_iv: "e996226deeb38f1a4a8273008f51c89a"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.199766 25058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 22:53:20.203037 25074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.204406 25058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:20.204571 25058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "52e3d8fb84b54488811f474e039e10a6"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "da804508ec8c2609828411914968db7a"
server_key_iv: "e996226deeb38f1a4a8273008f51c89a"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.204758 25058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:20.225737 25058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:20.229908 25058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:20.230207 25058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:20.231171 25058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:20.232546 25058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:20.232650 25058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.232790 25058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:20.232858 25058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.266185 25058 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:35385
I20260807 22:53:20.266191 25190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:35385 every 8 connection(s)
I20260807 22:53:20.267580 25058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:20.268523 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25058
I20260807 22:53:20.268850 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:53:20.269440 9995 external_mini_cluster.cc:1428] Setting key f0aa6f22c6a60c23a8ae3bbb6342f150
Aug 07 22:53:20 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:20.289045 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.270643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:53403 (local address 127.9.194.254:36625)
0807 22:53:20.271098 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.271104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.272250 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 22:53:20.274106 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.274114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.274207 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.274323 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:20.275868 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.277045 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.278540 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.279252 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.279398 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.282992 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:20.282997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:20.283000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:20.283032 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:20.285795 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.286716 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.286720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.286722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.286847 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.287334 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.287337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.287339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.287648 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:20.287762 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.288615 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.288772 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":215,"threads_started":1}
I20260807 22:53:20.290658 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.271151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:53403)
0807 22:53:20.272080 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:20.272134 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:20.273842 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:20.274545 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:20.274554 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:20.275059 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:20.275702 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.275721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.277181 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.277184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.278382 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.278393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.279511 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.279517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.279714 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.280386 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:20.280414 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:20.282755 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:20.285937 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:20.285948 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:20.285966 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:20.286598 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:20.286963 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:20.286970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:20.286977 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:20.287226 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:20.287768 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:20.287778 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:20.288214 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 22:53:20.288531 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 22:53:20.289113 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":437,"threads_started":1}
I20260807 22:53:20.292181 25191 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:20.292487 25191 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:20.293184 25191 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:20.294746 24854 ts_manager.cc:194] Registered new tserver with Master: 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
I20260807 22:53:20.296092 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:53403
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:53:20.331003 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:20.661643 25197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:20.661994 25197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:20.662134 25197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:20.669692 25197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:20.669797 25197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:20.669925 25197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:20.679401 25197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:20.680663 25197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:20.681931 25197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:20.694833 25203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:20.694967 25207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:20.694993 25205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:20.695981 25197 server_base.cc:1034] running on GCE node
I20260807 22:53:20.696547 25197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:20.697417 25197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:20.698637 25197 hybrid_clock.cc:648] HybridClock initialized: now 1786143200698598 us; error 47 us; skew 500 ppm
I20260807 22:53:20.698972 25197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:20 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:20.702083 25197 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:20.703591 25197 webserver.cc:460] Webserver started at http://127.9.194.195:42343/ using document root <none> and password file <none>
I20260807 22:53:20.704295 25197 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:20.704442 25197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:20.704732 25197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:20.707660 25197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "148571f6e09044a69712ed9bae3fe257"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "d1fa8ce05e4aef580515de81d8a20e52"
server_key_iv: "818514b69e0b3888f31a35d8859fb76c"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.708446 25197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "148571f6e09044a69712ed9bae3fe257"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "d1fa8ce05e4aef580515de81d8a20e52"
server_key_iv: "818514b69e0b3888f31a35d8859fb76c"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.713526 25197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 22:53:20.716980 25214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.718339 25197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:53:20.718519 25197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "148571f6e09044a69712ed9bae3fe257"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "d1fa8ce05e4aef580515de81d8a20e52"
server_key_iv: "818514b69e0b3888f31a35d8859fb76c"
server_key_version: "encryptionkey@0"
I20260807 22:53:20.718716 25197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:20.760516 25197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:20.764750 25197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:20.765072 25197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:20.766042 25197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:20.767537 25197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:20.767633 25197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.767777 25197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:20.767845 25197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:20.775944 25051 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:20.801064 25197 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42159
I20260807 22:53:20.801116 25330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42159 every 8 connection(s)
I20260807 22:53:20.802351 25197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:20.807607 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25197
I20260807 22:53:20.807807 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:53:20.808369 9995 external_mini_cluster.cc:1428] Setting key fbd0a6ca7460c5722f3ff4abf2882478
Aug 07 22:53:20 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:20.822798 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.804887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:37789 (local address 127.9.194.254:36625)
0807 22:53:20.805364 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.805369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.806313 (+ 944us) server_negotiation.cc:407] Connection header received
0807 22:53:20.808400 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.808405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.808505 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.808625 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:20.810464 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.811490 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.813020 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.813410 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.813541 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.816985 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:20.816990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:20.816993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:20.817025 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:20.819668 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.820388 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.820398 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.820401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.820521 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.820941 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.820946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.820950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.821222 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:20.821388 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.822154 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.822386 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":226,"threads_started":1}
I20260807 22:53:20.823935 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.805483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:37789)
0807 22:53:20.806151 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:20.806189 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:20.808069 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:20.808892 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:20.808904 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:20.809562 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:20.810304 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.810324 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.811635 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.811640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.812856 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.812868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.813640 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.813649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.813878 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.814528 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:20.814550 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:20.816768 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:20.819809 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:20.819819 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:20.819836 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:20.820264 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:20.820627 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:20.820632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:20.820636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:20.820828 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:20.821359 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:20.821367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:20.821728 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 22:53:20.821997 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 22:53:20.822540 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":272,"threads_started":1}
I20260807 22:53:20.825415 25331 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:20.825718 25331 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:20.826442 25331 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:20.827901 24854 ts_manager.cc:194] Registered new tserver with Master: 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159)
I20260807 22:53:20.828789 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:37789
I20260807 22:53:20.836588 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:53:20.853500 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.841099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.9.194.254:36625)
0807 22:53:20.841265 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.841269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.841795 (+ 526us) server_negotiation.cc:407] Connection header received
0807 22:53:20.841940 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.841943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.842008 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.842092 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:20.843489 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.844395 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.845676 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.846065 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.846156 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.848457 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:20.848462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:20.848465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:20.848495 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:20.851085 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.851831 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.851835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.851837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.851965 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:20.852271 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:20.852274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:20.852276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:20.852603 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:20.852780 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.853056 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.853229 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:53:20.857460 24854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:53:20.864665 24854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:53:20.893198 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.878093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:51404)
0807 22:53:20.883957 (+ 5864us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:20.884004 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:20.884234 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:20.885228 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:20.885235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:20.885263 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:20.886052 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.886068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.888120 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.888147 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.890490 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.890505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.892303 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.892309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.892533 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.892589 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:53:20.892716 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 22:53:20.892855 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5615,"mutex_wait_us":4,"thread_start_us":5423,"threads_started":1}
I20260807 22:53:20.894222 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.884606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.9.194.194:35385)
0807 22:53:20.885177 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.885185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.886135 (+ 950us) server_negotiation.cc:407] Connection header received
0807 22:53:20.886358 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.886370 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.886585 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.886798 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:20.887723 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.889144 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.891712 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.893099 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.893253 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.893431 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.893748 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.893918 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":288,"threads_started":1}
I20260807 22:53:20.894273 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.878567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.9.194.193:40459)
0807 22:53:20.879735 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.879748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.884119 (+ 4371us) server_negotiation.cc:407] Connection header received
0807 22:53:20.884457 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.884466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.884687 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.884875 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:20.886280 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.887979 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.890669 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.892090 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.892236 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.892465 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.893612 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.893784 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":312,"threads_started":1}
I20260807 22:53:20.894225 25350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.885246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:35385 (local address 127.0.0.1:46470)
0807 22:53:20.885989 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:20.886016 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:20.886151 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:20.887031 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:20.887039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:20.887063 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:20.887522 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.887536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.889329 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.889334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.891550 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.891565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.893320 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.893329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.893494 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.893527 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:53:20.893635 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:53:20.893764 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":392,"threads_started":1}
I20260807 22:53:20.895772 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.878300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42159 (local address 127.0.0.1:38558)
0807 22:53:20.883957 (+ 5657us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:20.884004 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:20.884192 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:20.885275 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:20.885279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:20.885308 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:20.886046 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.886058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.890651 (+ 4593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.890656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.892553 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:20.892566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.894985 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.894992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:20.895161 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.895190 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 22:53:20.895281 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:53:20.895408 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5486,"thread_start_us":5343,"threads_started":1}
I20260807 22:53:20.895905 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:20.878557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.9.194.195:42159)
0807 22:53:20.879214 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:20.879225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:20.884391 (+ 5166us) server_negotiation.cc:407] Connection header received
0807 22:53:20.884517 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:20.884524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:20.884768 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:20.884994 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:20.886332 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.887989 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.892715 (+ 4726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:20.894449 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:20.894595 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:20.894808 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:20.895324 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 22:53:20.895535 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":338,"threads_started":1}
I20260807 22:53:20.898523 24983 tablet_service.cc:1467] Processing CreateTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f (DEFAULT_TABLE table=test-table [id=6643ae1b81184faaabc3e20061c0bf10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:20.898898 25124 tablet_service.cc:1467] Processing CreateTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f (DEFAULT_TABLE table=test-table [id=6643ae1b81184faaabc3e20061c0bf10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:20.900375 24983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6f3ebcb94f54e439403fe02ae511b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:20.900362 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6f3ebcb94f54e439403fe02ae511b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:20.900251 25264 tablet_service.cc:1467] Processing CreateTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f (DEFAULT_TABLE table=test-table [id=6643ae1b81184faaabc3e20061c0bf10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:20.901585 25264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6f3ebcb94f54e439403fe02ae511b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:20.911784 25352 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Bootstrap starting.
I20260807 22:53:20.915416 25353 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Bootstrap starting.
I20260807 22:53:20.916055 25352 tablet_bootstrap.cc:654] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:20.917115 25354 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Bootstrap starting.
I20260807 22:53:20.917449 25352 log.cc:824] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:20.918807 25353 tablet_bootstrap.cc:654] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:20.920063 25353 log.cc:824] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:20.920235 25354 tablet_bootstrap.cc:654] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:20.920801 25352 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: No bootstrap required, opened a new log
I20260807 22:53:20.921315 25352 ts_tablet_manager.cc:1397] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 22:53:20.921450 25354 log.cc:824] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:20.923544 25353 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: No bootstrap required, opened a new log
I20260807 22:53:20.923810 25353 ts_tablet_manager.cc:1397] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:53:20.924579 25354 tablet_bootstrap.cc:492] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: No bootstrap required, opened a new log
I20260807 22:53:20.924836 25354 ts_tablet_manager.cc:1397] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:53:20.931721 25354 raft_consensus.cc:348] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.932216 25354 raft_consensus.cc:374] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:20.932374 25354 raft_consensus.cc:729] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148571f6e09044a69712ed9bae3fe257, State: Initialized, Role: FOLLOWER
I20260807 22:53:20.932157 25352 raft_consensus.cc:348] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.932647 25352 raft_consensus.cc:374] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:20.932775 25352 raft_consensus.cc:729] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e3d8fb84b54488811f474e039e10a6, State: Initialized, Role: FOLLOWER
I20260807 22:53:20.933102 25354 consensus_queue.cc:260] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.933492 25352 consensus_queue.cc:260] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.934274 25331 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:20.934357 25353 raft_consensus.cc:348] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.934762 25353 raft_consensus.cc:374] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:20.934779 25191 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:20.934845 25353 raft_consensus.cc:729] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d1d78776a74301b8cf4757700f4bde, State: Initialized, Role: FOLLOWER
I20260807 22:53:20.935191 25354 ts_tablet_manager.cc:1428] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 22:53:20.935472 25353 consensus_queue.cc:260] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:20.937245 25353 ts_tablet_manager.cc:1428] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260807 22:53:20.937281 25352 ts_tablet_manager.cc:1428] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Time spent starting tablet: real 0.016s user 0.012s sys 0.000s
W20260807 22:53:20.996178 25053 tablet.cc:2369] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:53:21.019774 25192 tablet.cc:2369] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:53:21.054026 25332 tablet.cc:2369] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:21.194244 25359 raft_consensus.cc:482] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:21.194598 25359 raft_consensus.cc:504] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:21.196408 25359 leader_election.cc:288] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459), 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159)
I20260807 22:53:21.205303 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.196839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:46137 (local address 127.9.194.193:40459)
0807 22:53:21.197126 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.197133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.197844 (+ 711us) server_negotiation.cc:407] Connection header received
0807 22:53:21.198105 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.198116 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.198180 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.198284 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:21.198859 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.200148 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.202723 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.204193 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.204345 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.204424 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.204916 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.205050 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:53:21.205299 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.196864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:54033 (local address 127.9.194.195:42159)
0807 22:53:21.197124 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.197132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.198267 (+ 1135us) server_negotiation.cc:407] Connection header received
0807 22:53:21.198444 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.198451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.198519 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.198591 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:21.199203 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.200420 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.202742 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.204156 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.204355 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.204426 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.204868 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.205008 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:53:21.205513 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.196993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42159 (local address 127.9.194.194:54033)
0807 22:53:21.198158 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:21.198172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:21.198307 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:21.198640 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:21.198649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:21.198680 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:21.199055 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:21.199063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.200543 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.200547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:21.202601 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:21.202613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.204455 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.204462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:21.204659 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.204708 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:53:21.204826 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 22:53:21.204995 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":56,"thread_start_us":849,"threads_started":1}
I20260807 22:53:21.205480 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.196907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.9.194.194:46137)
0807 22:53:21.197718 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:21.197736 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:21.197928 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:21.198322 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:21.198327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:21.198353 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:21.198697 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:21.198708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.200269 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.200272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:21.202528 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:21.202549 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.204475 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.204481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:21.204657 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.204710 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:53:21.204838 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 22:53:21.204989 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":46,"thread_start_us":454,"threads_started":1}
I20260807 22:53:21.206524 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6f3ebcb94f54e439403fe02ae511b9f" candidate_uuid: "52e3d8fb84b54488811f474e039e10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148571f6e09044a69712ed9bae3fe257" is_pre_election: true
I20260807 22:53:21.207007 25004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6f3ebcb94f54e439403fe02ae511b9f" candidate_uuid: "52e3d8fb84b54488811f474e039e10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d1d78776a74301b8cf4757700f4bde" is_pre_election: true
I20260807 22:53:21.207180 25285 raft_consensus.cc:2454] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52e3d8fb84b54488811f474e039e10a6 in term 0.
I20260807 22:53:21.207607 25004 raft_consensus.cc:2454] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52e3d8fb84b54488811f474e039e10a6 in term 0.
I20260807 22:53:21.207934 25076 leader_election.cc:302] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6; no voters:
I20260807 22:53:21.208391 25359 raft_consensus.cc:2781] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:53:21.208542 25359 raft_consensus.cc:482] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:53:21.208647 25359 raft_consensus.cc:3037] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:21.211050 25359 raft_consensus.cc:504] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:21.211714 25359 leader_election.cc:288] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [CANDIDATE]: Term 1 election: Requested vote from peers d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459), 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159)
I20260807 22:53:21.212148 25004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6f3ebcb94f54e439403fe02ae511b9f" candidate_uuid: "52e3d8fb84b54488811f474e039e10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d1d78776a74301b8cf4757700f4bde"
I20260807 22:53:21.212162 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6f3ebcb94f54e439403fe02ae511b9f" candidate_uuid: "52e3d8fb84b54488811f474e039e10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148571f6e09044a69712ed9bae3fe257"
I20260807 22:53:21.212364 25004 raft_consensus.cc:3037] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:21.212360 25285 raft_consensus.cc:3037] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:21.215602 25285 raft_consensus.cc:2454] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e3d8fb84b54488811f474e039e10a6 in term 1.
I20260807 22:53:21.215603 25004 raft_consensus.cc:2454] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e3d8fb84b54488811f474e039e10a6 in term 1.
I20260807 22:53:21.216003 25077 leader_election.cc:302] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52e3d8fb84b54488811f474e039e10a6, d7d1d78776a74301b8cf4757700f4bde; no voters:
I20260807 22:53:21.216279 25359 raft_consensus.cc:2781] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:21.216711 25359 raft_consensus.cc:686] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 1 LEADER]: Becoming Leader. State: Replica: 52e3d8fb84b54488811f474e039e10a6, State: Running, Role: LEADER
I20260807 22:53:21.217332 25359 consensus_queue.cc:237] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } }
I20260807 22:53:21.221858 24852 catalog_manager.cc:5495] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "52e3d8fb84b54488811f474e039e10a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } health_report { overall_health: HEALTHY } } }
I20260807 22:53:21.260074 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.254393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.9.194.194:35385)
0807 22:53:21.254596 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.254603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.254668 (+ 65us) server_negotiation.cc:407] Connection header received
0807 22:53:21.254961 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.254965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.255055 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.255210 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:53:21.255658 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.256396 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.257862 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.258303 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.258441 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.258923 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:53:21.259476 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:53:21.259634 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.259694 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.259777 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:53:21.268978 25285 raft_consensus.cc:1261] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Refusing update from remote peer 52e3d8fb84b54488811f474e039e10a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:21.269073 25004 raft_consensus.cc:1261] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Refusing update from remote peer 52e3d8fb84b54488811f474e039e10a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:21.270087 25363 consensus_queue.cc:1035] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:21.270615 25359 consensus_queue.cc:1035] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:21.282684 25366 mvcc.cc:205] Tried to move back new op lower bound from 7316042552387256320 to 7316042552190369792. Current Snapshot: MvccSnapshot[applied={T|T < 7316042552387256320}]
I20260807 22:53:21.292943 25369 mvcc.cc:205] Tried to move back new op lower bound from 7316042552387256320 to 7316042552190369792. Current Snapshot: MvccSnapshot[applied={T|T < 7316042552387256320}]
I20260807 22:53:21.303340 24853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:21.303718 24853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:21.307793 24853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Sending DeleteTablet for 3 replicas of tablet f6f3ebcb94f54e439403fe02ae511b9f
I20260807 22:53:21.309046 25264 tablet_service.cc:1514] Processing DeleteTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:21 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:38558
I20260807 22:53:21.309173 24983 tablet_service.cc:1514] Processing DeleteTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:21 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51404
I20260807 22:53:21.309688 25124 tablet_service.cc:1514] Processing DeleteTablet for tablet f6f3ebcb94f54e439403fe02ae511b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:21 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:46470
I20260807 22:53:21.310712 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24917
I20260807 22:53:21.321499 25377 tablet_replica.cc:330] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: stopping tablet replica
I20260807 22:53:21.322081 25376 tablet_replica.cc:330] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: stopping tablet replica
I20260807 22:53:21.322574 25375 tablet_replica.cc:330] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: stopping tablet replica
I20260807 22:53:21.323045 25377 raft_consensus.cc:2229] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:53:21.323112 25375 raft_consensus.cc:2229] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:21.323349 25376 raft_consensus.cc:2229] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:21.323796 25375 raft_consensus.cc:2258] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:21.324939 25377 raft_consensus.cc:2258] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:21.326161 25375 ts_tablet_manager.cc:1905] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:21.329167 25377 ts_tablet_manager.cc:1905] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:21.324296 25376 raft_consensus.cc:2258] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:21.333243 25375 ts_tablet_manager.cc:1918] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:21.333463 25375 log.cc:1196] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/f6f3ebcb94f54e439403fe02ae511b9f
I20260807 22:53:21.333757 25376 ts_tablet_manager.cc:1905] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:21.334021 25375 ts_tablet_manager.cc:1939] T f6f3ebcb94f54e439403fe02ae511b9f P 148571f6e09044a69712ed9bae3fe257: Deleting consensus metadata
Aug 07 22:53:21 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:53:21.336814 25377 ts_tablet_manager.cc:1918] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:21.337124 25377 log.cc:1196] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/f6f3ebcb94f54e439403fe02ae511b9f
I20260807 22:53:21.337796 25377 ts_tablet_manager.cc:1939] T f6f3ebcb94f54e439403fe02ae511b9f P 52e3d8fb84b54488811f474e039e10a6: Deleting consensus metadata
I20260807 22:53:21.339767 24840 catalog_manager.cc:4841] TS 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): tablet f6f3ebcb94f54e439403fe02ae511b9f (table test-table [id=6643ae1b81184faaabc3e20061c0bf10]) successfully deleted
I20260807 22:53:21.340018 24839 catalog_manager.cc:4841] TS 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): tablet f6f3ebcb94f54e439403fe02ae511b9f (table test-table [id=6643ae1b81184faaabc3e20061c0bf10]) successfully deleted
I20260807 22:53:21.343228 25376 ts_tablet_manager.cc:1918] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:21.343416 25376 log.cc:1196] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/f6f3ebcb94f54e439403fe02ae511b9f
I20260807 22:53:21.344115 25376 ts_tablet_manager.cc:1939] T f6f3ebcb94f54e439403fe02ae511b9f P d7d1d78776a74301b8cf4757700f4bde: Deleting consensus metadata
I20260807 22:53:21.345264 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.312228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.9.194.193:40459)
0807 22:53:21.312452 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.312461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.321799 (+ 9338us) server_negotiation.cc:407] Connection header received
0807 22:53:21.322086 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.322095 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.322198 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.322299 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:21.324458 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.327400 (+ 2942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.329577 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.330233 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.330349 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.337117 (+ 6768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:21.337131 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:21.337148 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:21.337195 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:21.341435 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.342451 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.342462 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.342467 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.342655 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.343121 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.343127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.343131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.343677 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:21.343996 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.344479 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.344737 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:53:21.345968 25046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:21.346048 24840 catalog_manager.cc:4841] TS d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459): tablet f6f3ebcb94f54e439403fe02ae511b9f (table test-table [id=6643ae1b81184faaabc3e20061c0bf10]) successfully deleted
I20260807 22:53:21.442462 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24917
I20260807 22:53:21.462199 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25058
Aug 07 22:53:21 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:53:21.484230 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.463325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.9.194.194:35385)
0807 22:53:21.463551 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.463558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.463579 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:53:21.463648 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.463654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.463742 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.463860 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:21.465088 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.466318 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.467630 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.468329 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.468469 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.477382 (+ 8913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:21.477402 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:21.477423 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:21.477479 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:21.481343 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.482239 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.482245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.482248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.482415 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.482817 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.482820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.482822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.483177 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:21.483593 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.483760 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.483931 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:53:21.485009 25186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:21.598755 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25058
I20260807 22:53:21.619836 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25197
Aug 07 22:53:21 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:53:21.635277 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:21.620618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.9.194.195:42159)
0807 22:53:21.620812 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:21.620821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:21.620843 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:53:21.621090 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:21.621098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:21.621207 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:21.621311 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:21.622670 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.623621 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.624758 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:21.625173 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:21.625270 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:21.629780 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:21.629794 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:21.629810 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:21.629845 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:21.632583 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.633275 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.633282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.633284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.633418 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:21.633774 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:21.633779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:21.633786 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:21.634183 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:21.634418 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:21.634684 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:53:21.634884 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:53:21.635941 25326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:21.726747 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25197
I20260807 22:53:21.745754 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 24821
I20260807 22:53:21.746964 24884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:21.859539 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 24821
I20260807 22:53:21.884246 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--webserver_interface=127.9.194.254
--webserver_port=43195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:22.226416 25384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:22.226730 25384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:22.226918 25384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:22.234179 25384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:22.234319 25384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:22.234421 25384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:22.234512 25384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:22.234596 25384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:22.244860 25384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=43195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:22.246163 25384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:22.247493 25384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:22.276628 25391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:22.276650 25390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:22.276727 25393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:22.277051 25384 server_base.cc:1034] running on GCE node
I20260807 22:53:22.278407 25384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:22.280107 25384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:22.281348 25384 hybrid_clock.cc:648] HybridClock initialized: now 1786143202281283 us; error 53 us; skew 500 ppm
I20260807 22:53:22.281764 25384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:22 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143202, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:22.285472 25384 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:22.287158 25384 webserver.cc:460] Webserver started at http://127.9.194.254:43195/ using document root <none> and password file <none>
I20260807 22:53:22.288009 25384 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:22.288172 25384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:22.294232 25384 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:53:22.297204 25401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:22.298573 25384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:53:22.298761 25384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a2e3df33c004a58af245c9db8bd74f7"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "95d5ebc820b038844dce0fa291dcee25"
server_key_iv: "28ed5c511fee9baa2163a1669aa6526b"
server_key_version: "encryptionkey@0"
I20260807 22:53:22.299932 25384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:22.311813 25384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:22.316272 25384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:22.316541 25384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:22.335505 25384 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:36625
I20260807 22:53:22.335585 25454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:36625 every 8 connection(s)
I20260807 22:53:22.337026 25384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:22.339298 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25384
I20260807 22:53:22.340149 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:40459
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=40617
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:22.344295 25455 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:53:22.347625 25455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap starting.
I20260807 22:53:22.367614 25455 log.cc:824] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:22.379863 25455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:53:22.380407 25455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap complete.
I20260807 22:53:22.389778 25455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:22.390642 25455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Initialized, Role: FOLLOWER
I20260807 22:53:22.391290 25455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:22.391597 25455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:22.391727 25455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:22.391919 25455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:53:22.395910 25455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:22.396456 25455 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2e3df33c004a58af245c9db8bd74f7; no voters:
I20260807 22:53:22.397116 25455 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:53:22.397251 25459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:53:22.398473 25459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Running, Role: LEADER
I20260807 22:53:22.399145 25459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:22.399608 25455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:22.401880 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:22.402384 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:22.403030 25461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2e3df33c004a58af245c9db8bd74f7. Latest consensus state: current_term: 2 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:22.403443 25461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:22.405779 25464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:22.415585 25464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:22.417557 25464 catalog_manager.cc:1259] Loaded cluster ID: e75b7e716980498f8607449016e65722
I20260807 22:53:22.417750 25464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:22.421896 25464 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 22:53:22.434110 25476 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:53:22.440460 25464 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 22:53:22.447559 25464 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:22.448907 25464 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Loaded TSK: 0
I20260807 22:53:22.450042 25464 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:53:22.718647 25457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:22.718976 25457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:22.719120 25457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:22.726554 25457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:22.726665 25457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:22.726827 25457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:22.736766 25457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:40459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=40617
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:22.738178 25457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:22.739677 25457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:22.756280 25485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:22.756253 25484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:22.757143 25487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:22.757437 25457 server_base.cc:1034] running on GCE node
I20260807 22:53:22.758167 25457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:22.759233 25457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:22.760478 25457 hybrid_clock.cc:648] HybridClock initialized: now 1786143202760415 us; error 77 us; skew 500 ppm
I20260807 22:53:22.760833 25457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:22 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143202, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:22.764303 25457 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:22.765971 25457 webserver.cc:460] Webserver started at http://127.9.194.193:40617/ using document root <none> and password file <none>
I20260807 22:53:22.766888 25457 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:22.767112 25457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:22.773998 25457 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 22:53:22.777550 25494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:22.779170 25457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:53:22.779383 25457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "d7d1d78776a74301b8cf4757700f4bde"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "8c94db0018e11adfff70fc51fef374d2"
server_key_iv: "37ee91ff5063d11b8c3b172481dfbf07"
server_key_version: "encryptionkey@0"
I20260807 22:53:22.780494 25457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:22.809973 25457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:22.815177 25457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:22.815583 25457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:22.816725 25457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:22.818306 25457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:22.818421 25457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:22.818573 25457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:22.818681 25457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:22.853953 25457 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:40459
I20260807 22:53:22.853982 25612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:40459 every 8 connection(s)
I20260807 22:53:22.855708 25457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:22.858348 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25457
I20260807 22:53:22.859252 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:35385
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=44803
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:22 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143202, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:22.890422 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:22.859149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:44177)
0807 22:53:22.864095 (+ 4946us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:22.864155 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:22.867008 (+ 2853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:22.868788 (+ 1780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:22.868802 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:22.869640 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:22.870488 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:22.870517 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:22.873322 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:22.873327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:22.875312 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:22.875329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:22.876988 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:22.876997 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:22.877326 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:22.878221 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:22.878256 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:22.881232 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:22.886061 (+ 4829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:22.886068 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:22.886081 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:22.886623 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:22.887205 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:22.887208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:22.887211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:22.887393 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:22.888486 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:22.888491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:22.888791 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 22:53:22.889025 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 22:53:22.889420 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4596,"thread_start_us":4300,"threads_started":1}
I20260807 22:53:22.891228 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:22.858971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:44177 (local address 127.9.194.254:36625)
0807 22:53:22.859827 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:22.859850 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:22.864352 (+ 4502us) server_negotiation.cc:407] Connection header received
0807 22:53:22.867695 (+ 3343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:22.867710 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:22.868234 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:22.868562 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:22.870820 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:22.873122 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:22.875516 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:22.876699 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:22.876955 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:22.881563 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:22.881577 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:22.881602 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:22.881657 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:22.885902 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:22.886794 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:22.886820 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:22.886831 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:22.887076 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:22.887544 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:22.887553 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:22.887558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:22.888339 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:22.888551 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:22.889325 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 22:53:22.889836 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":309,"threads_started":1}
I20260807 22:53:22.894115 25613 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:22.894425 25613 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:22.895128 25613 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:22.897329 25420 ts_manager.cc:194] Registered new tserver with Master: d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459)
I20260807 22:53:22.898756 25420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:44177
W20260807 22:53:23.202008 25617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:23.202323 25617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:23.202490 25617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:23.209857 25617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:23.210000 25617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:23.210175 25617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:23.219746 25617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:35385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=44803
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:23.221072 25617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:23.222518 25617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:23.235742 25626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:23.235790 25623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:23.235761 25624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:23.236510 25617 server_base.cc:1034] running on GCE node
I20260807 22:53:23.237115 25617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:23.238107 25617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:23.239333 25617 hybrid_clock.cc:648] HybridClock initialized: now 1786143203239284 us; error 61 us; skew 500 ppm
I20260807 22:53:23.239673 25617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:23 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:23.242760 25617 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:23.244288 25617 webserver.cc:460] Webserver started at http://127.9.194.194:44803/ using document root <none> and password file <none>
I20260807 22:53:23.244987 25617 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:23.245133 25617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:23.250311 25617 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:53:23.253041 25634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.254345 25617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:53:23.254518 25617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "52e3d8fb84b54488811f474e039e10a6"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "da804508ec8c2609828411914968db7a"
server_key_iv: "e996226deeb38f1a4a8273008f51c89a"
server_key_version: "encryptionkey@0"
I20260807 22:53:23.255501 25617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:23.267522 25617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:23.271358 25617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:23.271659 25617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:23.272619 25617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:23.273916 25617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:23.274035 25617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.274158 25617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:23.274241 25617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.307931 25617 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:35385
I20260807 22:53:23.308015 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:35385 every 8 connection(s)
I20260807 22:53:23.309399 25617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:23.313983 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25617
I20260807 22:53:23.314594 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:42159
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=42343
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:53:23 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:23.330650 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.311948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:48985 (local address 127.9.194.254:36625)
0807 22:53:23.312135 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.312139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.313304 (+ 1165us) server_negotiation.cc:407] Connection header received
0807 22:53:23.315221 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.315230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.315329 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.315471 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:23.317014 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.318159 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.319774 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.320616 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.320757 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.324452 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:23.324462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:23.324466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:23.324512 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:23.327290 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.328249 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.328253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.328255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.328380 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.328887 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.328890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.328892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.329184 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:23.329297 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.330223 (+ 926us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.330403 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:53:23.332305 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.312450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:48985)
0807 22:53:23.313136 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.313176 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.314945 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.315758 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.315767 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.316261 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:23.316826 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.316844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.318316 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.318319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.319623 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.319634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.320842 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.320847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.321045 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.321745 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:23.321766 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:23.324232 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:23.327450 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:23.327462 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:23.327480 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:23.328127 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:23.328497 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:23.328505 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:23.328511 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:23.328777 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:23.329313 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:23.329323 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:23.329808 (+ 485us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.330135 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.330713 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":261,"threads_started":1}
I20260807 22:53:23.333793 25756 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:23.334112 25756 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:23.334820 25756 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:23.336346 25420 ts_manager.cc:194] Registered new tserver with Master: 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
I20260807 22:53:23.337164 25420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:48985
W20260807 22:53:23.659605 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:23.659869 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:23.660027 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:23.667158 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:23.667258 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:23.667496 25759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:23.677973 25759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:42159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=42343
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:23.679288 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:23.680670 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:23.693655 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:23.693629 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:23.693742 25770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:23.694545 25759 server_base.cc:1034] running on GCE node
I20260807 22:53:23.695070 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:23.695994 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:23.697190 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786143203697142 us; error 37 us; skew 500 ppm
I20260807 22:53:23.697520 25759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:23 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:23.700675 25759 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:23.702126 25759 webserver.cc:460] Webserver started at http://127.9.194.195:42343/ using document root <none> and password file <none>
I20260807 22:53:23.702781 25759 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:23.702948 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:23.708415 25759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:53:23.711305 25777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.712828 25759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:53:23.712993 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "148571f6e09044a69712ed9bae3fe257"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "d1fa8ce05e4aef580515de81d8a20e52"
server_key_iv: "818514b69e0b3888f31a35d8859fb76c"
server_key_version: "encryptionkey@0"
I20260807 22:53:23.713925 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:23.733403 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:23.737329 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:23.737613 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:23.738662 25759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:23.740106 25759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:23.740223 25759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.740356 25759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:23.740456 25759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:23.773788 25759 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42159
I20260807 22:53:23.773877 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42159 every 8 connection(s)
I20260807 22:53:23.775290 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:23.785785 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25759
Aug 07 22:53:23 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:23.795368 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.777935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:42323 (local address 127.9.194.254:36625)
0807 22:53:23.778137 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.778141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.779387 (+ 1246us) server_negotiation.cc:407] Connection header received
0807 22:53:23.781283 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.781288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.781354 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.781441 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:23.782848 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.783622 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.785206 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.785863 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.786026 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.789662 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:23.789671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:23.789674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:23.789709 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:23.792436 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.793129 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.793136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.793141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.793271 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.793693 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.793699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.793702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.793968 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:23.794117 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.794803 (+ 686us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.795041 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:53:23.796625 25900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.778454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:42323)
0807 22:53:23.779223 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.779268 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.781034 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.781642 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.781650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.782164 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:23.782707 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.782724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.783746 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.783750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.785078 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.785090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.786104 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.786109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.786296 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.786991 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:23.787034 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:23.789415 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:23.792577 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:23.792586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:23.792601 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:23.793013 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:23.793378 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:23.793383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:23.793387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:23.793582 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:23.794091 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:23.794100 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:23.794439 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.794706 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.795223 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":268,"threads_started":1}
I20260807 22:53:23.798199 25898 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36625
I20260807 22:53:23.798503 25898 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:23.799506 25898 heartbeater.cc:511] Master 127.9.194.254:36625 requested a full tablet report, sending...
I20260807 22:53:23.801363 25420 ts_manager.cc:194] Registered new tserver with Master: 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159)
I20260807 22:53:23.801623 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.786847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35516 (local address 127.9.194.254:36625)
0807 22:53:23.787755 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.787760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.787776 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:53:23.787822 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.787826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.787874 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.787990 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:23.789542 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.790525 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.792094 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.792555 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.792684 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.795468 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:23.795476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:23.795482 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:23.795527 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:23.798608 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.799470 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.799475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.799479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.799596 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:23.800015 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:23.800018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:23.800020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:23.800344 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:23.800587 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.800902 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.801152 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":618,"threads_started":1}
I20260807 22:53:23.802551 25420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:42323
I20260807 22:53:23.804361 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:53:23.810796 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.805310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35524 (local address 127.9.194.254:36625)
0807 22:53:23.805473 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.805477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.806020 (+ 543us) server_negotiation.cc:407] Connection header received
0807 22:53:23.806222 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.806227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.806341 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.806499 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:53:23.806847 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.807634 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.808949 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.809342 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.809437 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.809859 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:53:23.810326 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:53:23.810448 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.810548 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.810609 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:53:23.811671 25420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:53:23.819084 25420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:53:23.843312 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.831613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.9.194.195:42159)
0807 22:53:23.834584 (+ 2971us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.834593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.834625 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:53:23.834724 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.834732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.834978 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.835185 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:23.836393 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.837906 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.840460 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.841739 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.841863 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.842119 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.842733 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.842932 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2387,"thread_start_us":2179,"threads_started":1}
I20260807 22:53:23.843853 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.832000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.9.194.194:35385)
0807 22:53:23.833119 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.833127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.833154 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:53:23.833229 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.833235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.833485 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.835540 (+ 2055us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:23.836498 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.838096 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.840960 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.842445 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.842583 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.842787 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.843338 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.843536 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":1441,"threads_started":1}
I20260807 22:53:23.843943 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.831293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:35385 (local address 127.0.0.1:46490)
0807 22:53:23.832531 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.832546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.832633 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.835508 (+ 2875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.835513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.835540 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:23.836261 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.836278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.838605 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.838610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.840794 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.840809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.842932 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.842938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.843119 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.843160 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.843252 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.843508 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":28,"thread_start_us":917,"threads_started":1}
I20260807 22:53:23.844524 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.833607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.9.194.193:40459)
0807 22:53:23.834471 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.834483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.836557 (+ 2074us) server_negotiation.cc:407] Connection header received
0807 22:53:23.836711 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.836720 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.836970 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.837154 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:23.837733 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.839370 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.841553 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.842823 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.842955 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.843137 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.844046 (+ 909us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.844225 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":533,"threads_started":1}
I20260807 22:53:23.844552 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.833047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:51424)
0807 22:53:23.836441 (+ 3394us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.836458 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.836575 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.837164 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.837169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.837195 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:23.837552 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.837563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.839533 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.839539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.841400 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.841411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.843310 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.843317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.843526 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.843556 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.843651 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.843783 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3268,"thread_start_us":3056,"threads_started":1}
I20260807 22:53:23.847272 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.831096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42159 (local address 127.0.0.1:38578)
0807 22:53:23.832041 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.832083 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.832286 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.835241 (+ 2955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.835252 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.835297 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:23.835986 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.836006 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.838088 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.838095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.840302 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.840314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.842257 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.842263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.842462 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.842528 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.846761 (+ 4233us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.846896 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":24,"thread_start_us":1134,"threads_started":1}
I20260807 22:53:23.848333 25687 tablet_service.cc:1467] Processing CreateTablet for tablet e9013568d14d4f97b91710d657fea739 (DEFAULT_TABLE table=test-table [id=f3bff60021dd49dc9218bdad3ab6c21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:23.848356 25544 tablet_service.cc:1467] Processing CreateTablet for tablet e9013568d14d4f97b91710d657fea739 (DEFAULT_TABLE table=test-table [id=f3bff60021dd49dc9218bdad3ab6c21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:23.849488 25687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9013568d14d4f97b91710d657fea739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:23.849752 25544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9013568d14d4f97b91710d657fea739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:23.850945 25826 tablet_service.cc:1467] Processing CreateTablet for tablet e9013568d14d4f97b91710d657fea739 (DEFAULT_TABLE table=test-table [id=f3bff60021dd49dc9218bdad3ab6c21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:23.852335 25826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9013568d14d4f97b91710d657fea739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:23.866050 25916 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Bootstrap starting.
I20260807 22:53:23.868357 25918 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Bootstrap starting.
I20260807 22:53:23.870620 25916 tablet_bootstrap.cc:654] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:23.871711 25918 tablet_bootstrap.cc:654] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:23.872160 25916 log.cc:824] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:23.872246 25917 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Bootstrap starting.
I20260807 22:53:23.872991 25918 log.cc:824] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:23.875798 25918 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: No bootstrap required, opened a new log
I20260807 22:53:23.875972 25916 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: No bootstrap required, opened a new log
I20260807 22:53:23.876098 25918 ts_tablet_manager.cc:1397] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 22:53:23.876255 25916 ts_tablet_manager.cc:1397] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 22:53:23.876760 25917 tablet_bootstrap.cc:654] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:23.878002 25917 log.cc:824] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:23.881121 25917 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: No bootstrap required, opened a new log
I20260807 22:53:23.881385 25917 ts_tablet_manager.cc:1397] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 22:53:23.884416 25918 raft_consensus.cc:348] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.884845 25918 raft_consensus.cc:374] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:23.884948 25918 raft_consensus.cc:729] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148571f6e09044a69712ed9bae3fe257, State: Initialized, Role: FOLLOWER
I20260807 22:53:23.885610 25918 consensus_queue.cc:260] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.886699 25898 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:23.886696 25916 raft_consensus.cc:348] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.887105 25916 raft_consensus.cc:374] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:23.887192 25916 raft_consensus.cc:729] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d1d78776a74301b8cf4757700f4bde, State: Initialized, Role: FOLLOWER
I20260807 22:53:23.887825 25916 consensus_queue.cc:260] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.888211 25918 ts_tablet_manager.cc:1428] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260807 22:53:23.889262 25613 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:23.889756 25916 ts_tablet_manager.cc:1428] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260807 22:53:23.891840 25917 raft_consensus.cc:348] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.892320 25917 raft_consensus.cc:374] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:23.892424 25917 raft_consensus.cc:729] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e3d8fb84b54488811f474e039e10a6, State: Initialized, Role: FOLLOWER
I20260807 22:53:23.893102 25917 consensus_queue.cc:260] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.894196 25756 heartbeater.cc:503] Master 127.9.194.254:36625 was elected leader, sending a full tablet report...
I20260807 22:53:23.894913 25917 ts_tablet_manager.cc:1428] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260807 22:53:23.971800 25923 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:23.972123 25923 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.973861 25923 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
I20260807 22:53:23.981215 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.974249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:44855 (local address 127.9.194.195:42159)
0807 22:53:23.974475 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.974481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.975412 (+ 931us) server_negotiation.cc:407] Connection header received
0807 22:53:23.975706 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.975714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.975787 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.975858 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:23.976509 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.977353 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.979340 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.980169 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.980348 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.980435 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.980804 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.980938 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:53:23.981225 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.974472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58789 (local address 127.9.194.194:35385)
0807 22:53:23.974716 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:23.974725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:23.975572 (+ 847us) server_negotiation.cc:407] Connection header received
0807 22:53:23.975756 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:23.975760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:23.975817 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:23.975914 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:23.976586 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.977369 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.979309 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.980175 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.980350 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.980426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:23.980821 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 22:53:23.980950 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:53:23.981386 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.974351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42159 (local address 127.9.194.193:44855)
0807 22:53:23.975299 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.975315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.975518 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.975901 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.975905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.975929 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:23.976362 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.976374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.977481 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.977486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.979159 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.979172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.980382 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.980389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.980617 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.980652 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.980775 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.980944 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":135,"thread_start_us":566,"threads_started":1}
I20260807 22:53:23.981756 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:23.974448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:35385 (local address 127.9.194.193:58789)
0807 22:53:23.975417 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:23.975478 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:23.975612 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:23.975953 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:23.975957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:23.975982 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:23.976388 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.976396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.977531 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.977538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.979151 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:23.979163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:23.980419 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:23.980425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:23.980634 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:23.980657 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:53:23.980776 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:53:23.980938 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":89,"thread_start_us":607,"threads_started":1}
I20260807 22:53:23.982486 25850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9013568d14d4f97b91710d657fea739" candidate_uuid: "d7d1d78776a74301b8cf4757700f4bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148571f6e09044a69712ed9bae3fe257" is_pre_election: true
I20260807 22:53:23.982787 25710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9013568d14d4f97b91710d657fea739" candidate_uuid: "d7d1d78776a74301b8cf4757700f4bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e3d8fb84b54488811f474e039e10a6" is_pre_election: true
I20260807 22:53:23.983001 25850 raft_consensus.cc:2454] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d1d78776a74301b8cf4757700f4bde in term 0.
I20260807 22:53:23.983361 25710 raft_consensus.cc:2454] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d1d78776a74301b8cf4757700f4bde in term 0.
I20260807 22:53:23.983772 25497 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148571f6e09044a69712ed9bae3fe257, d7d1d78776a74301b8cf4757700f4bde; no voters:
I20260807 22:53:23.984172 25923 raft_consensus.cc:2781] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:53:23.984324 25923 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:53:23.984437 25923 raft_consensus.cc:3037] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:23.986840 25923 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.987455 25923 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 1 election: Requested vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
I20260807 22:53:23.987793 25850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9013568d14d4f97b91710d657fea739" candidate_uuid: "d7d1d78776a74301b8cf4757700f4bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148571f6e09044a69712ed9bae3fe257"
I20260807 22:53:23.987952 25710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9013568d14d4f97b91710d657fea739" candidate_uuid: "d7d1d78776a74301b8cf4757700f4bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e3d8fb84b54488811f474e039e10a6"
I20260807 22:53:23.987977 25850 raft_consensus.cc:3037] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:23.988159 25710 raft_consensus.cc:3037] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:23.990227 25850 raft_consensus.cc:2454] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d1d78776a74301b8cf4757700f4bde in term 1.
I20260807 22:53:23.990370 25710 raft_consensus.cc:2454] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d1d78776a74301b8cf4757700f4bde in term 1.
I20260807 22:53:23.990623 25497 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148571f6e09044a69712ed9bae3fe257, d7d1d78776a74301b8cf4757700f4bde; no voters:
I20260807 22:53:23.990906 25923 raft_consensus.cc:2781] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:23.991338 25923 raft_consensus.cc:686] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 LEADER]: Becoming Leader. State: Replica: d7d1d78776a74301b8cf4757700f4bde, State: Running, Role: LEADER
I20260807 22:53:23.991884 25923 consensus_queue.cc:237] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:23.996186 25420 catalog_manager.cc:5495] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d1d78776a74301b8cf4757700f4bde (127.9.194.193). New cstate: current_term: 1 leader_uuid: "d7d1d78776a74301b8cf4757700f4bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } health_report { overall_health: HEALTHY } } }
W20260807 22:53:24.027014 25899 tablet.cc:2369] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:24.035259 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.029738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.9.194.193:40459)
0807 22:53:24.029958 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:24.029965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:24.029985 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:53:24.030247 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:24.030252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:24.030331 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:24.030495 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:53:24.030888 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.031632 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.033111 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.033547 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.033692 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:24.034162 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:53:24.034669 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:53:24.034843 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:24.034893 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 22:53:24.034973 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:53:24.043335 25710 raft_consensus.cc:1261] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Refusing update from remote peer d7d1d78776a74301b8cf4757700f4bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:24.043355 25850 raft_consensus.cc:1261] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Refusing update from remote peer d7d1d78776a74301b8cf4757700f4bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:24.044384 25927 consensus_queue.cc:1035] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:24.044730 25923 consensus_queue.cc:1035] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:24.074313 25420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:24.074599 25420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:24.078282 25420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Sending DeleteTablet for 3 replicas of tablet e9013568d14d4f97b91710d657fea739
I20260807 22:53:24.079501 25826 tablet_service.cc:1514] Processing DeleteTablet for tablet e9013568d14d4f97b91710d657fea739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:24 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:38578
I20260807 22:53:24.079707 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25457
I20260807 22:53:24.079651 25687 tablet_service.cc:1514] Processing DeleteTablet for tablet e9013568d14d4f97b91710d657fea739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:24 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:46490
I20260807 22:53:24.081139 25544 tablet_service.cc:1514] Processing DeleteTablet for tablet e9013568d14d4f97b91710d657fea739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:24 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:51424
I20260807 22:53:24.091188 25941 tablet_replica.cc:330] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: stopping tablet replica
I20260807 22:53:24.091825 25941 raft_consensus.cc:2229] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:24.092447 25941 raft_consensus.cc:2258] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:24.094764 25941 ts_tablet_manager.cc:1905] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:24.096100 25942 tablet_replica.cc:330] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: stopping tablet replica
I20260807 22:53:24.096629 25942 raft_consensus.cc:2229] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:24.097231 25942 raft_consensus.cc:2258] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:24.099566 25942 ts_tablet_manager.cc:1905] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:24.102465 25941 ts_tablet_manager.cc:1918] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:24.102681 25941 log.cc:1196] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/e9013568d14d4f97b91710d657fea739
I20260807 22:53:24.103451 25941 ts_tablet_manager.cc:1939] T e9013568d14d4f97b91710d657fea739 P 148571f6e09044a69712ed9bae3fe257: Deleting consensus metadata
I20260807 22:53:24.105559 25403 catalog_manager.cc:4841] TS 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): tablet e9013568d14d4f97b91710d657fea739 (table test-table [id=f3bff60021dd49dc9218bdad3ab6c21e]) successfully deleted
I20260807 22:53:24.107143 25942 ts_tablet_manager.cc:1918] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:24.107336 25942 log.cc:1196] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/e9013568d14d4f97b91710d657fea739
I20260807 22:53:24.108001 25942 ts_tablet_manager.cc:1939] T e9013568d14d4f97b91710d657fea739 P 52e3d8fb84b54488811f474e039e10a6: Deleting consensus metadata
I20260807 22:53:24.110270 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.080548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.9.194.193:40459)
0807 22:53:24.080708 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:24.080713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:24.080759 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:53:24.081744 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:24.081754 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:24.081858 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:24.081984 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:24.084892 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.086514 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.088955 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.089888 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.090040 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:24.101702 (+ 11662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:24.101718 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:24.101736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:24.101791 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:24.106235 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.107381 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.107391 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.107395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.107700 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.108243 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.108248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.108252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.108739 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:24.109115 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:24.109555 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 22:53:24.109831 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:53:24.111649 25608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:24.112221 25404 catalog_manager.cc:4841] TS 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): tablet e9013568d14d4f97b91710d657fea739 (table test-table [id=f3bff60021dd49dc9218bdad3ab6c21e]) successfully deleted
I20260807 22:53:24.213588 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25457
I20260807 22:53:24.233855 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25617
W20260807 22:53:24.233796 25404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:40459 (error 108)
I20260807 22:53:24.235841 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.235393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:51450)
0807 22:53:24.235591 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.235709 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 22:53:24.236213 25404 catalog_manager.cc:4584] TS d7d1d78776a74301b8cf4757700f4bde (127.9.194.193:40459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9013568d14d4f97b91710d657fea739: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
I20260807 22:53:24.247869 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.234733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.9.194.194:35385)
0807 22:53:24.234981 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:24.234987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:24.235087 (+ 100us) server_negotiation.cc:407] Connection header received
0807 22:53:24.235283 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:24.235287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:24.235351 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:24.235489 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:24.237028 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.237800 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.239183 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.239798 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.239895 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:24.242508 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:24.242522 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:24.242536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:24.242566 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:24.245316 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.246020 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.246027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.246030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.246152 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.246499 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.246502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.246504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.246852 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:24.247102 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:24.247398 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:53:24.247606 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:53:24.248570 25750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:24.289887 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.289384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:51460)
0807 22:53:24.289624 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.289718 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 22:53:24.290854 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.290512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:51470)
0807 22:53:24.290654 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.290746 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:53:24.338219 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25617
I20260807 22:53:24.351522 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.351007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:49954)
0807 22:53:24.351197 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.351281 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:53:24.352948 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.352529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:49956)
0807 22:53:24.352706 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.352798 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:53:24.360241 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25759
I20260807 22:53:24.373153 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.361026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.9.194.195:42159)
0807 22:53:24.361199 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:24.361203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:24.361222 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:53:24.361396 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:24.361400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:24.361465 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:24.361548 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:24.362923 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.363726 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.364982 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:24.365354 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:24.365443 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:24.367899 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:24.367913 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:24.367929 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:24.367971 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:24.370593 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.371249 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.371260 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.371264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.371464 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:24.371804 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:24.371807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:24.371809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:24.372158 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:24.372374 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:24.372633 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 22:53:24.372804 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:53:24.373857 25893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:24.433495 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.432890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:49972)
0807 22:53:24.433165 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.433280 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 22:53:24.434753 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:24.434309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:40459 (local address 127.0.0.1:49988)
0807 22:53:24.434529 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:24.434621 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 22:53:24.462572 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25759
I20260807 22:53:24.481202 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 25384
I20260807 22:53:24.482344 25450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:24.597487 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 25384
I20260807 22:53:24.622764 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--webserver_interface=127.9.194.254
--webserver_port=43195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:24.967118 25949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:24.967459 25949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:24.967619 25949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:24.974889 25949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:24.975024 25949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:24.975111 25949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:24.975215 25949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:24.975291 25949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:24.985143 25949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=43195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:24.986460 25949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:24.987882 25949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:24.999614 25954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:24.999571 25955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:24.999643 25957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:25.000609 25949 server_base.cc:1034] running on GCE node
I20260807 22:53:25.001330 25949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:25.002676 25949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:25.003875 25949 hybrid_clock.cc:648] HybridClock initialized: now 1786143205003817 us; error 42 us; skew 500 ppm
I20260807 22:53:25.004197 25949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:25 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143205, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:25.007737 25949 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:25.009234 25949 webserver.cc:460] Webserver started at http://127.9.194.254:43195/ using document root <none> and password file <none>
I20260807 22:53:25.009938 25949 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:25.010084 25949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:25.015559 25949 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:25.018386 25964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:25.019739 25949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:25.019920 25949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a2e3df33c004a58af245c9db8bd74f7"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "95d5ebc820b038844dce0fa291dcee25"
server_key_iv: "28ed5c511fee9baa2163a1669aa6526b"
server_key_version: "encryptionkey@0"
I20260807 22:53:25.020900 25949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:25.044829 25949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:25.049239 25949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:25.049515 25949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:25.068228 25949 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:36625
I20260807 22:53:25.068259 26018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:36625 every 8 connection(s)
I20260807 22:53:25.069832 25949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:25.076763 26019 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:53:25.079491 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 25949
I20260807 22:53:25.080343 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:40459
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=40617
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:25.080397 26019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap starting.
I20260807 22:53:25.104588 26019 log.cc:824] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:25.127022 26019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:53:25.127760 26019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7: Bootstrap complete.
I20260807 22:53:25.141167 26019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:25.142527 26019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Initialized, Role: FOLLOWER
I20260807 22:53:25.143283 26019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:25.143591 26019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:25.143749 26019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:25.143937 26019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:53:25.149746 26019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:25.150353 26019 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2e3df33c004a58af245c9db8bd74f7; no voters:
I20260807 22:53:25.151057 26019 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:53:25.151213 26023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:53:25.152659 26023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [term 3 LEADER]: Becoming Leader. State: Replica: 8a2e3df33c004a58af245c9db8bd74f7, State: Running, Role: LEADER
I20260807 22:53:25.153182 26023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } }
I20260807 22:53:25.153537 26019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:25.156023 26026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2e3df33c004a58af245c9db8bd74f7. Latest consensus state: current_term: 3 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:25.156070 26024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8a2e3df33c004a58af245c9db8bd74f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2e3df33c004a58af245c9db8bd74f7" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36625 } } }
I20260807 22:53:25.156495 26024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:25.156499 26026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2e3df33c004a58af245c9db8bd74f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:25.169731 26031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:25.179790 26031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:25.182749 26031 catalog_manager.cc:1259] Loaded cluster ID: e75b7e716980498f8607449016e65722
I20260807 22:53:25.182969 26031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 22:53:25.185268 26031 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe1979dd9ad google::LogMessage::Fail() at ??:0
@ 0x7fe1979e088a google::LogMessage::SendToLog() at ??:0
I20260807 22:53:25.191687 26042 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fe1979dd4de google::LogMessage::Flush() at ??:0
@ 0x7fe1979dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe1ad410512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe1ad40d976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe1ad495610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe1ad495439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe19a629345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe19a62c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe19a62c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe19a607eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe19d8e16db start_thread at ??:0
@ 0x7fe19687761f clone at ??:0
W20260807 22:53:25.501230 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:25.501515 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:25.501669 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:25.509186 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:25.509305 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:25.509518 26021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:25.519466 26021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:40459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=40617
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:25.520795 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:25.522146 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:25.536363 26063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:25.536434 26066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:25.536470 26064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:25.537777 26021 server_base.cc:1034] running on GCE node
I20260807 22:53:25.538439 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:25.539376 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:25.540628 26021 hybrid_clock.cc:648] HybridClock initialized: now 1786143205540584 us; error 38 us; skew 500 ppm
I20260807 22:53:25.540951 26021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:25 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143205, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:25.544150 26021 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:25.545643 26021 webserver.cc:460] Webserver started at http://127.9.194.193:40617/ using document root <none> and password file <none>
I20260807 22:53:25.546368 26021 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:25.546514 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:25.552006 26021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:53:25.554930 26073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:25.556288 26021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:25.556463 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "d7d1d78776a74301b8cf4757700f4bde"
format_stamp: "Formatted at 2026-08-07 22:53:19 on dist-test-slave-5k54"
server_key: "8c94db0018e11adfff70fc51fef374d2"
server_key_iv: "37ee91ff5063d11b8c3b172481dfbf07"
server_key_version: "encryptionkey@0"
I20260807 22:53:25.557446 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:25.594679 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:25.599086 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:25.599392 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:25.600473 26021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:25.602429 26080 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 22:53:25.608533 26021 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:53:25.608732 26021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 22:53:25.608894 26021 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:53:25.612190 26021 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:53:25.612324 26021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:25.612643 26080 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Bootstrap starting.
I20260807 22:53:25.641113 26080 log.cc:824] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:25.653681 26021 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:40459
I20260807 22:53:25.653740 26196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:40459 every 8 connection(s)
I20260807 22:53:25.654721 26080 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:53:25.655002 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:25.655359 26080 tablet_bootstrap.cc:492] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Bootstrap complete.
I20260807 22:53:25.655606 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26021
I20260807 22:53:25.656606 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:35385
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=44803
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:25.657315 26080 ts_tablet_manager.cc:1397] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Time spent bootstrapping tablet: real 0.045s user 0.024s sys 0.019s
I20260807 22:53:25.669963 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:25.658586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:45775)
0807 22:53:25.668166 (+ 9580us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:25.668574 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9173,"thread_start_us":8949,"threads_started":1}
W20260807 22:53:25.670753 26197 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:36625 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:36625: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
I20260807 22:53:25.670132 26080 raft_consensus.cc:348] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:25.671505 26080 raft_consensus.cc:729] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d1d78776a74301b8cf4757700f4bde, State: Initialized, Role: FOLLOWER
I20260807 22:53:25.671938 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:25.671465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:57911)
0807 22:53:25.671711 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:25.671798 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 22:53:25.672230 26080 consensus_queue.cc:260] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:25.673053 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:25.672567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:46627)
0807 22:53:25.672726 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:25.672856 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:53:25.673395 26197 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:53:25.674166 26080 ts_tablet_manager.cc:1428] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260807 22:53:25.674325 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:25.673918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:38091)
0807 22:53:25.674087 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:25.674177 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:53:25.675278 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:25.674871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:36699)
0807 22:53:25.675032 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:25.675123 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 22:53:26.001143 26200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:26.001461 26200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:26.001556 26200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:26.008781 26200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:26.008906 26200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:26.009023 26200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:26.018693 26200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:35385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=44803
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:26.019971 26200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:26.021277 26200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:26.034526 26206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:26.034552 26207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:26.034463 26209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:26.035383 26200 server_base.cc:1034] running on GCE node
I20260807 22:53:26.036005 26200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:26.036875 26200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:26.038141 26200 hybrid_clock.cc:648] HybridClock initialized: now 1786143206038091 us; error 50 us; skew 500 ppm
I20260807 22:53:26.038502 26200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:26 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143206, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:26.041961 26200 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:26.043459 26200 webserver.cc:460] Webserver started at http://127.9.194.194:44803/ using document root <none> and password file <none>
I20260807 22:53:26.044210 26200 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:26.044356 26200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:26.050168 26200 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:53:26.053109 26216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.054522 26200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:53:26.054718 26200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "52e3d8fb84b54488811f474e039e10a6"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "da804508ec8c2609828411914968db7a"
server_key_iv: "e996226deeb38f1a4a8273008f51c89a"
server_key_version: "encryptionkey@0"
I20260807 22:53:26.055783 26200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:26.070302 26200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:26.075102 26200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:26.075397 26200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:26.076458 26200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:26.077919 26200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:26.078053 26200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.078195 26200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:26.078287 26200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.115338 26200 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:35385
I20260807 22:53:26.115483 26335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:35385 every 8 connection(s)
I20260807 22:53:26.116935 26200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:26.122057 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.120147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:57669)
0807 22:53:26.120939 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.121183 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":303,"threads_started":1}
W20260807 22:53:26.122766 26336 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:36625 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:36625: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
I20260807 22:53:26.122824 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26200
I20260807 22:53:26.123711 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:42159
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:36625
--webserver_port=42343
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:37625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:53:26.123777 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.123370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:44179)
0807 22:53:26.123567 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.123637 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:53:26.124764 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.124425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:37511)
0807 22:53:26.124570 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.124652 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 22:53:26.125082 26336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:53:26.464076 26340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:26.464352 26340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:26.464449 26340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:26.471606 26340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:26.471711 26340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:26.471827 26340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:26.481664 26340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:42159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=42343
--tserver_master_addrs=127.9.194.254:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:26.482990 26340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:26.484441 26340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:26.497984 26346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:26.497885 26347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:26.497920 26349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:26.498917 26340 server_base.cc:1034] running on GCE node
I20260807 22:53:26.499722 26340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:26.501348 26340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:26.502611 26340 hybrid_clock.cc:648] HybridClock initialized: now 1786143206502534 us; error 65 us; skew 500 ppm
I20260807 22:53:26.503042 26340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:26 dist-test-slave-5k54 krb5kdc[24808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143206, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:26.506419 26340 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:26.508013 26340 webserver.cc:460] Webserver started at http://127.9.194.195:42343/ using document root <none> and password file <none>
I20260807 22:53:26.508736 26340 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:26.508879 26340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:26.514489 26340 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 22:53:26.517431 26357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.518890 26340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:53:26.519135 26340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "148571f6e09044a69712ed9bae3fe257"
format_stamp: "Formatted at 2026-08-07 22:53:20 on dist-test-slave-5k54"
server_key: "d1fa8ce05e4aef580515de81d8a20e52"
server_key_iv: "818514b69e0b3888f31a35d8859fb76c"
server_key_version: "encryptionkey@0"
I20260807 22:53:26.520218 26340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:26.539671 26340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:26.544389 26340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:26.544634 26340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:26.545631 26340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:26.547014 26340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:26.547115 26340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.547201 26340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:26.547257 26340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:26.581796 26340 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:42159
I20260807 22:53:26.581926 26482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:42159 every 8 connection(s)
I20260807 22:53:26.583343 26340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:26.588399 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.586448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:43231)
0807 22:53:26.587221 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.587534 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":343,"threads_started":1}
W20260807 22:53:26.589104 26483 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:36625 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:36625: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
I20260807 22:53:26.589939 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.589631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:43837)
0807 22:53:26.589767 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.589835 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 22:53:26.590750 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.590468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:37585)
0807 22:53:26.590582 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.590652 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 22:53:26.590978 26483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:53:26.592197 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26340
I20260807 22:53:26.677076 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:26.676256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:35539)
0807 22:53:26.676717 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:26.676827 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":219,"threads_started":1}
I20260807 22:53:27.056201 26488 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:27.056535 26488 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:27.058167 26488 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
Aug 07 22:53:27 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143205, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
Aug 07 22:53:27 dist-test-slave-5k54 krb5kdc[24808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143205, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:53:27.084491 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.058606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:35385 (local address 127.9.194.193:38793)
0807 22:53:27.061356 (+ 2750us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:27.061376 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:27.061516 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:27.064484 (+ 2968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:27.064491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:27.065631 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:27.066818 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:27.066846 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.069225 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.069231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:27.071747 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:27.071762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.073435 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.073440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:27.073578 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:27.074243 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:27.074261 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:27.077251 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:27.080755 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:27.080759 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:27.080761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:27.081267 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:27.081741 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:27.081753 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:27.081756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:27.081916 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:27.082930 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:27.082933 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:27.083103 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 22:53:27.083478 (+ 375us) client_negotiation.cc:240] Negotiation successful
0807 22:53:27.084019 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2606,"mutex_wait_us":379,"spinlock_wait_cycles":28544,"thread_start_us":2326,"threads_started":1}
I20260807 22:53:27.084482 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.058876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:35715 (local address 127.9.194.195:42159)
0807 22:53:27.060760 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:27.060782 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:27.060837 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:53:27.063307 (+ 2470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:27.063320 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:27.063853 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:27.064215 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:27.067045 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.068972 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.071910 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.072938 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.073133 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:27.077211 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:27.077222 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:27.077241 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:27.077296 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:27.080446 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:27.081417 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:27.081424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:27.081430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:27.081585 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:27.082037 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:27.082049 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:27.082051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:27.082561 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:27.082699 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:27.083710 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0807 22:53:27.084097 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1647,"thread_start_us":1498,"threads_started":1}
I20260807 22:53:27.084564 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.058794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:38793 (local address 127.9.194.194:35385)
0807 22:53:27.060996 (+ 2202us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:27.061013 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:27.061484 (+ 471us) server_negotiation.cc:407] Connection header received
0807 22:53:27.063737 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:27.063747 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:27.064165 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:27.064492 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:27.067056 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.068979 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.071910 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.072938 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.073133 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:27.077514 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:27.077521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:27.077531 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:27.077573 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:27.080606 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:27.081421 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:27.081428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:27.081434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:27.081592 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:27.082065 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:27.082069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:27.082071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:27.082793 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:27.082967 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:27.083715 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 22:53:27.084120 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2030,"thread_start_us":1910,"threads_started":1}
I20260807 22:53:27.085661 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.058608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:42159 (local address 127.9.194.193:35715)
0807 22:53:27.058966 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:27.059019 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:27.059704 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:27.064318 (+ 4614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:27.064326 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:27.065631 (+ 1305us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:27.066818 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:27.066846 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.069230 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.069234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:27.071746 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:27.071762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:27.073159 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:27.073165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:27.073358 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:27.074076 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:27.074101 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:27.076848 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:27.080603 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:27.080612 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:27.080626 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:27.081260 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:27.081715 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:27.081720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:27.081724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:27.081910 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:27.082731 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:27.082739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:27.083157 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 22:53:27.083472 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 22:53:27.085326 (+ 1854us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1746,"spinlock_wait_cycles":22912}
W20260807 22:53:27.088544 26076 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
W20260807 22:53:27.088819 26075 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
I20260807 22:53:27.088984 26075 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7d1d78776a74301b8cf4757700f4bde; no voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6
I20260807 22:53:27.089440 26488 raft_consensus.cc:2726] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:53:27.126564 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.125670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:49749)
0807 22:53:27.126169 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:27.126311 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":230,"threads_started":1}
I20260807 22:53:27.592372 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.591687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:34935)
0807 22:53:27.592105 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:27.592195 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":212,"threads_started":1}
I20260807 22:53:27.678843 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:27.678103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:52539)
0807 22:53:27.678592 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:27.678678 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":248,"threads_started":1}
I20260807 22:53:28.128322 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:28.127634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:44387)
0807 22:53:28.128059 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:28.128143 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":218,"threads_started":1}
I20260807 22:53:28.594158 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:28.593372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:47813)
0807 22:53:28.593839 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:28.593932 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":238,"threads_started":1}
I20260807 22:53:28.680658 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:28.679860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:59057)
0807 22:53:28.680373 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:28.680463 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":232,"threads_started":1}
I20260807 22:53:29.130266 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:29.129426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:47909)
0807 22:53:29.129970 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:29.130076 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":280,"threads_started":1}
I20260807 22:53:29.184908 26500 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:29.185143 26500 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:29.185801 26500 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
W20260807 22:53:29.186836 26076 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
W20260807 22:53:29.187106 26075 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
I20260807 22:53:29.187207 26075 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7d1d78776a74301b8cf4757700f4bde; no voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6
I20260807 22:53:29.187552 26500 raft_consensus.cc:2726] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:53:29.595902 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:29.595102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:58423)
0807 22:53:29.595635 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:29.595734 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":235,"threads_started":1}
I20260807 22:53:29.682502 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:29.681684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:55219)
0807 22:53:29.682208 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:29.682301 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":273,"threads_started":1}
I20260807 22:53:30.132310 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:30.131390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:50235)
0807 22:53:30.131967 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:30.132075 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":265,"threads_started":1}
I20260807 22:53:30.597600 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:30.596876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:44929)
0807 22:53:30.597309 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:30.597399 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":212,"threads_started":1}
I20260807 22:53:30.684573 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:30.683682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:40751)
0807 22:53:30.684300 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:30.684388 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":380,"threads_started":1}
I20260807 22:53:31.133906 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:31.133219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:42127)
0807 22:53:31.133660 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:31.133741 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260807 22:53:31.599475 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:31.598618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:58785)
0807 22:53:31.599115 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:31.599205 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":300,"threads_started":1}
I20260807 22:53:31.686214 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:31.685513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:36837)
0807 22:53:31.685971 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:31.686052 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":254,"threads_started":1}
I20260807 22:53:31.994935 26510 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:31.995159 26510 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:31.995947 26510 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
W20260807 22:53:31.996986 26075 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
W20260807 22:53:31.997174 26076 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
I20260807 22:53:31.997273 26076 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7d1d78776a74301b8cf4757700f4bde; no voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6
I20260807 22:53:31.997490 26510 raft_consensus.cc:2726] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:53:32.135541 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:32.134794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:58171)
0807 22:53:32.135247 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:32.135335 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260807 22:53:32.601100 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:32.600412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:44433)
0807 22:53:32.600848 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:32.600935 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":227,"threads_started":1}
I20260807 22:53:32.687932 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:32.687122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:49637)
0807 22:53:32.687617 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:32.687710 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":220,"threads_started":1}
I20260807 22:53:33.137225 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:33.136431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:45027)
0807 22:53:33.136934 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:33.137038 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":230,"threads_started":1}
I20260807 22:53:33.602788 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:33.601959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:49765)
0807 22:53:33.602479 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:33.602577 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":243,"threads_started":1}
I20260807 22:53:33.689553 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:33.688787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:57515)
0807 22:53:33.689306 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:33.689390 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":310,"threads_started":1}
I20260807 22:53:34.138926 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:34.138197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:44269)
0807 22:53:34.138596 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:34.138677 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":209,"threads_started":1}
I20260807 22:53:34.604380 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:34.603687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:48521)
0807 22:53:34.604115 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:34.604196 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":217,"threads_started":1}
I20260807 22:53:34.691236 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:34.690486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:44605)
0807 22:53:34.690956 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:34.691048 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
I20260807 22:53:35.140616 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:35.139861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:49515)
0807 22:53:35.140369 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:35.140459 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":264,"threads_started":1}
I20260807 22:53:35.605989 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:35.605275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:54663)
0807 22:53:35.605744 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:35.605828 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":224,"threads_started":1}
I20260807 22:53:35.692953 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:35.692263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:37335)
0807 22:53:35.692711 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:35.692791 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260807 22:53:36.142222 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:36.141527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:44437)
0807 22:53:36.141942 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:36.142037 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":219,"threads_started":1}
I20260807 22:53:36.607764 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:36.606926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:59161)
0807 22:53:36.607473 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:36.607565 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":299,"threads_started":1}
I20260807 22:53:36.694850 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:36.693945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:34089)
0807 22:53:36.694487 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:36.694576 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":288,"threads_started":1}
I20260807 22:53:36.794157 26527 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:36.794464 26527 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:36.795265 26527 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
W20260807 22:53:36.796268 26075 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
W20260807 22:53:36.796470 26076 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
I20260807 22:53:36.796566 26076 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7d1d78776a74301b8cf4757700f4bde; no voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6
I20260807 22:53:36.796793 26527 raft_consensus.cc:2726] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:53:37.144062 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:37.143170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:45255)
0807 22:53:37.143767 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:37.143862 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":332,"threads_started":1}
I20260807 22:53:37.609419 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:37.608672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:50053)
0807 22:53:37.609166 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:37.609262 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":258,"threads_started":1}
I20260807 22:53:37.696586 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:37.695821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:43089)
0807 22:53:37.696300 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:37.696382 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":270,"threads_started":1}
I20260807 22:53:38.145749 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:38.145043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:59325)
0807 22:53:38.145496 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:38.145578 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":230,"threads_started":1}
I20260807 22:53:38.611075 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:38.610348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:57781)
0807 22:53:38.610795 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:38.610877 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":240,"threads_started":1}
I20260807 22:53:38.698377 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:38.697546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:38879)
0807 22:53:38.698090 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:38.698201 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":249,"threads_started":1}
I20260807 22:53:38.917327 26534 raft_consensus.cc:482] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:38.917557 26534 raft_consensus.cc:504] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148571f6e09044a69712ed9bae3fe257" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 42159 } } peers { permanent_uuid: "52e3d8fb84b54488811f474e039e10a6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 35385 } } peers { permanent_uuid: "d7d1d78776a74301b8cf4757700f4bde" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 40459 } }
I20260807 22:53:38.918195 26534 leader_election.cc:288] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159), 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385)
W20260807 22:53:38.919222 26076 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52e3d8fb84b54488811f474e039e10a6 (127.9.194.194:35385): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
W20260807 22:53:38.919466 26075 leader_election.cc:341] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 148571f6e09044a69712ed9bae3fe257 (127.9.194.195:42159): Not found: Tablet not found: e9013568d14d4f97b91710d657fea739
I20260807 22:53:38.919581 26075 leader_election.cc:302] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7d1d78776a74301b8cf4757700f4bde; no voters: 148571f6e09044a69712ed9bae3fe257, 52e3d8fb84b54488811f474e039e10a6
I20260807 22:53:38.919791 26534 raft_consensus.cc:2726] T e9013568d14d4f97b91710d657fea739 P d7d1d78776a74301b8cf4757700f4bde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:53:39.147315 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:39.146616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:56801)
0807 22:53:39.147057 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:39.147151 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":219,"threads_started":1}
I20260807 22:53:39.612723 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:39.612019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:50787)
0807 22:53:39.612472 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:39.612570 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":228,"threads_started":1}
I20260807 22:53:39.700031 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:39.699229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:53413)
0807 22:53:39.699732 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:39.699824 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":316,"threads_started":1}
I20260807 22:53:40.148972 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:40.148227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:35785)
0807 22:53:40.148675 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:40.148760 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":227,"threads_started":1}
I20260807 22:53:40.614358 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:40.613576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:58581)
0807 22:53:40.614084 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:40.614163 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":277,"threads_started":1}
I20260807 22:53:40.701947 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:40.701179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:40579)
0807 22:53:40.701672 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:40.701760 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":250,"threads_started":1}
I20260807 22:53:41.151021 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.150182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.194:37435)
0807 22:53:41.150663 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:41.150852 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":256,"threads_started":1}
I20260807 22:53:41.597946 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26021
I20260807 22:53:41.613714 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.599161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.9.194.193:40459)
0807 22:53:41.599842 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:41.599852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:41.599886 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:53:41.599985 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:41.599998 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:41.600321 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:41.600507 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:41.602280 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.603251 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.604543 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.605183 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.605289 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:41.608033 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:41.608044 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:41.608058 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:41.608089 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:41.611076 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.611770 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.611777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.611779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.611953 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.612281 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.612284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.612286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.612633 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:41.612867 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:41.613143 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:53:41.613384 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":320,"threads_started":1}
I20260807 22:53:41.615121 26191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:41.616664 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.615557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.195:33443)
0807 22:53:41.616198 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:41.616385 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":268,"threads_started":1}
I20260807 22:53:41.705701 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.704554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36625 (local address 127.9.194.193:43585)
0807 22:53:41.705168 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:41.705357 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":309,"threads_started":1}
I20260807 22:53:41.705789 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26021
I20260807 22:53:41.727536 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26200
I20260807 22:53:41.740419 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.728286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.9.194.194:35385)
0807 22:53:41.728792 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:41.728799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:41.728819 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:53:41.728880 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:41.728884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:41.728943 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:41.729049 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:41.730382 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.731130 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.732441 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.732875 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.732972 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:41.735400 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:41.735404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:41.735407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:41.735488 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:41.738089 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.738727 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.738731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.738733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.738871 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.739188 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.739192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.739194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.739542 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:41.739761 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:41.739949 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 22:53:41.740116 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":296,"threads_started":1}
I20260807 22:53:41.741176 26331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:41.825526 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26200
I20260807 22:53:41.844870 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26340
I20260807 22:53:41.858486 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:41.845814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.9.194.195:42159)
0807 22:53:41.846402 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:41.846407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:41.846429 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:53:41.846485 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:41.846489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:41.846561 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:41.846665 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:41.848282 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.849112 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.850332 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:41.850757 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:41.850853 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:41.853510 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:41.853515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:41.853519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:41.853550 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:41.856248 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.856900 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.856903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.856906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.857062 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:41.857368 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:41.857371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:41.857373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:41.857612 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:41.857820 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:41.858034 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 22:53:41.858197 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":327,"threads_started":1}
I20260807 22:53:41.859205 26478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:41.944132 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26340
2026-08-07T22:53:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24733 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:53:42 dist-test-slave-5k54 krb5kdc[26552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:53:42 dist-test-slave-5k54 krb5kdc[26552](info): set up 2 sockets
Aug 07 22:53:42 dist-test-slave-5k54 krb5kdc[26552](info): commencing operation
krb5kdc: starting...
W20260807 22:53:43.105242 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:53:43 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:53:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:53:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:53:43.378492 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:43.716320 26567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:43.716601 26567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:43.716701 26567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:43.723917 26567 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:53:43.724009 26567 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:53:43.724045 26567 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:53:43.724095 26567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:53:43.724146 26567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:43.724232 26567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:43.724277 26567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:53:43.724329 26567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:53:43.734238 26567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:43.735476 26567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:43.736779 26567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:43.749071 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:43.749001 26574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:43.749048 26576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:43.749678 26567 server_base.cc:1034] running on GCE node
I20260807 22:53:43.750532 26567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:43.751929 26567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:43.753144 26567 hybrid_clock.cc:648] HybridClock initialized: now 1786143223753084 us; error 28 us; skew 500 ppm
I20260807 22:53:43.753484 26567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:43 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:43.760861 26567 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:53:43.762283 26567 webserver.cc:460] Webserver started at http://127.9.194.254:45985/ using document root <none> and password file <none>
I20260807 22:53:43.762992 26567 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:43.763125 26567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:43.763502 26567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:43.766480 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "8a9e67bee9fe488ba2c5b74915883421"
format_stamp: "Formatted at 2026-08-07 22:53:43 on dist-test-slave-5k54"
server_key: "2e78e037bb636923cd4e2a53a2f4a942"
server_key_iv: "44bc0df3f2dd74c3f0b60cce97df1881"
server_key_version: "encryptionkey@0"
I20260807 22:53:43.767267 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "8a9e67bee9fe488ba2c5b74915883421"
format_stamp: "Formatted at 2026-08-07 22:53:43 on dist-test-slave-5k54"
server_key: "2e78e037bb636923cd4e2a53a2f4a942"
server_key_iv: "44bc0df3f2dd74c3f0b60cce97df1881"
server_key_version: "encryptionkey@0"
I20260807 22:53:43.772389 26567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:53:43.775841 26583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:43.777100 26567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:53:43.777285 26567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a9e67bee9fe488ba2c5b74915883421"
format_stamp: "Formatted at 2026-08-07 22:53:43 on dist-test-slave-5k54"
server_key: "2e78e037bb636923cd4e2a53a2f4a942"
server_key_iv: "44bc0df3f2dd74c3f0b60cce97df1881"
server_key_version: "encryptionkey@0"
I20260807 22:53:43.777518 26567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:43.789867 26567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:43.793618 26567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:43.793886 26567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:43.811713 26567 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:45469
I20260807 22:53:43.811666 26634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:45469 every 8 connection(s)
I20260807 22:53:43.813066 26567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:53:43.814255 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26567
I20260807 22:53:43.814507 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:53:43.815340 9995 external_mini_cluster.cc:1428] Setting key 0452ca1d91494309e764007988de8368
I20260807 22:53:43.817893 26635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:53:43 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:43.831573 26635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap starting.
I20260807 22:53:43.835769 26635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:43.837049 26635 log.cc:824] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:43.840564 26635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: No bootstrap required, opened a new log
I20260807 22:53:43.841073 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:43.817514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.9.194.254:45469)
0807 22:53:43.818335 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:43.818346 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:43.818386 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:53:43.820167 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:43.820189 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:43.820576 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:43.820898 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:43.822791 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:43.824259 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:43.826233 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:43.826950 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:43.827105 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:43.832255 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:43.832264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:43.832284 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:43.832332 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:43.836479 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:43.837207 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:43.837215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:43.837222 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:43.837424 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:43.837911 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:43.837914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:43.837916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:43.838534 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:43.838805 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:43.839237 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 22:53:43.839727 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":615,"thread_start_us":416,"threads_started":1}
I20260807 22:53:43.848740 26635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:53:43.849095 26635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:43.849169 26635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Initialized, Role: FOLLOWER
I20260807 22:53:43.849648 26635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:53:43.849941 26635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:53:43.850136 26635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:53:43.850329 26635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:43.852628 26635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:53:43.853078 26635 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e67bee9fe488ba2c5b74915883421; no voters:
I20260807 22:53:43.853644 26635 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:53:43.853786 26641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:43.854533 26641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 LEADER]: Becoming Leader. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Running, Role: LEADER
I20260807 22:53:43.855055 26641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:53:43.855762 26635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:53:43.857808 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:53:43.858198 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:43.858536 26643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e67bee9fe488ba2c5b74915883421. Latest consensus state: current_term: 1 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:53:43.858891 26643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:53:43.865201 26647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:53:43.872243 26647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:53:43.881493 26647 catalog_manager.cc:1347] Generated new cluster ID: f0b91b432696475181e3a406c408ada9
I20260807 22:53:43.881774 26647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:53:43.899313 26647 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:53:43.901304 26647 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:53:43.911752 26647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Generated new TSK 0
I20260807 22:53:43.912817 26647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:53:43.985558 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:44.319734 26665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:44.319984 26665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:44.320160 26665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:44.327317 26665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:44.327481 26665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:44.327687 26665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:53:44.337289 26665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:44.338536 26665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:44.339903 26665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:44.352979 26670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:44.352970 26673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:44.353013 26671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:44.354010 26665 server_base.cc:1034] running on GCE node
I20260807 22:53:44.354592 26665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:44.355561 26665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:44.356787 26665 hybrid_clock.cc:648] HybridClock initialized: now 1786143224356677 us; error 70 us; skew 500 ppm
I20260807 22:53:44.357101 26665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:44 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:44.360227 26665 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:53:44.361649 26665 webserver.cc:460] Webserver started at http://127.9.194.193:37145/ using document root <none> and password file <none>
I20260807 22:53:44.362332 26665 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:44.362483 26665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:44.362794 26665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:44.365855 26665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "bf9364736a479a9cecd85d694143361e"
server_key_iv: "2bd5e32148c48f5cdf5381371377058a"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.366626 26665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "bf9364736a479a9cecd85d694143361e"
server_key_iv: "2bd5e32148c48f5cdf5381371377058a"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.371640 26665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:53:44.374974 26680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.376267 26665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:53:44.376439 26665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "bf9364736a479a9cecd85d694143361e"
server_key_iv: "2bd5e32148c48f5cdf5381371377058a"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.376637 26665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:44.390427 26665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:44.394124 26665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:44.394411 26665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:44.395316 26665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:44.396615 26665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:44.396719 26665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.396875 26665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:44.396961 26665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.430601 26665 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39815
I20260807 22:53:44.430641 26800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39815 every 8 connection(s)
I20260807 22:53:44.432049 26665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:53:44.434119 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26665
I20260807 22:53:44.434350 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:53:44.435174 9995 external_mini_cluster.cc:1428] Setting key 95b94e59406db0b6c6f277436b691c34
Aug 07 22:53:44 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:44.453889 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:44.434775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:48169 (local address 127.9.194.254:45469)
0807 22:53:44.435317 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:44.435324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:44.436527 (+ 1203us) server_negotiation.cc:407] Connection header received
0807 22:53:44.438551 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:44.438560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:44.438650 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:44.438777 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:44.440326 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.441483 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.442971 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.443737 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.443903 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:44.447681 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:44.447690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:44.447695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:44.447738 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:44.450833 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:44.451572 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:44.451577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:44.451581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:44.451711 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:44.452153 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:44.452157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:44.452163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:44.452421 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:44.452589 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:44.453286 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 22:53:44.453528 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260807 22:53:44.454802 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:44.435397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:48169)
0807 22:53:44.436338 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:44.436396 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:44.438286 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:44.438978 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:44.438986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:44.439571 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:44.440143 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:44.440161 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.441629 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.441633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:44.442830 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:44.442841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.443957 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.443962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:44.444155 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:44.444861 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:44.444882 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:44.447416 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:44.450980 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:44.450989 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:44.451003 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:44.451415 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:44.451823 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:44.451828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:44.451832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:44.452033 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:44.452558 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:44.452566 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:44.452907 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 22:53:44.453173 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 22:53:44.453683 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":313,"threads_started":1}
I20260807 22:53:44.456493 26801 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:53:44.456841 26801 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:44.457581 26801 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:53:44.459968 26600 ts_manager.cc:194] Registered new tserver with Master: 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815)
I20260807 22:53:44.462227 26600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:48169
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:53:44.498431 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:44.829342 26807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:44.829604 26807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:44.829705 26807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:44.836853 26807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:44.836972 26807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:44.837085 26807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:53:44.846576 26807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:44.847792 26807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:44.849071 26807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:44.861804 26812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:44.862046 26813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:44.862234 26816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:44.863251 26807 server_base.cc:1034] running on GCE node
I20260807 22:53:44.863875 26807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:44.864713 26807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:44.865921 26807 hybrid_clock.cc:648] HybridClock initialized: now 1786143224865863 us; error 53 us; skew 500 ppm
I20260807 22:53:44.866184 26807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:44 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:44.869640 26807 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:53:44.874325 26807 webserver.cc:460] Webserver started at http://127.9.194.194:36453/ using document root <none> and password file <none>
I20260807 22:53:44.875172 26807 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:44.875315 26807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:44.875779 26807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:44.879010 26807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "e266ab6347834bcf8379e819178980c1"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "ef7a69c02918e09696525b99ae7997fc"
server_key_iv: "80f7d9136ca2b9016e2cdee79ae29bae"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.879814 26807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "e266ab6347834bcf8379e819178980c1"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "ef7a69c02918e09696525b99ae7997fc"
server_key_iv: "80f7d9136ca2b9016e2cdee79ae29bae"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.885221 26807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:53:44.888635 26823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.889930 26807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:53:44.890087 26807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "e266ab6347834bcf8379e819178980c1"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "ef7a69c02918e09696525b99ae7997fc"
server_key_iv: "80f7d9136ca2b9016e2cdee79ae29bae"
server_key_version: "encryptionkey@0"
I20260807 22:53:44.890285 26807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:44.913185 26807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:44.917114 26807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:44.917407 26807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:44.918362 26807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:44.919744 26807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:44.919819 26807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.919979 26807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:44.920029 26807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:44.954865 26807 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:41409
I20260807 22:53:44.954962 26944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:41409 every 8 connection(s)
I20260807 22:53:44.956367 26807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:53:44.965008 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26807
I20260807 22:53:44.965197 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:53:44.965684 9995 external_mini_cluster.cc:1428] Setting key c55043ea0332cabcbc7871b38453bdd6
Aug 07 22:53:44 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:44.978130 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:44.958918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59541 (local address 127.9.194.254:45469)
0807 22:53:44.959367 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:44.959372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:44.960231 (+ 859us) server_negotiation.cc:407] Connection header received
0807 22:53:44.962039 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:44.962044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:44.962132 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:44.962260 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:44.963903 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.964663 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.966702 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.967196 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.967344 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:44.971106 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:44.971115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:44.971122 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:44.971172 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:44.974735 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:44.975489 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:44.975497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:44.975504 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:44.975664 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:44.976103 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:44.976111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:44.976117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:44.976500 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:44.976653 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:44.977364 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 22:53:44.977633 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":252,"threads_started":1}
I20260807 22:53:44.978718 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:44.959339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:59541)
0807 22:53:44.960079 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:44.960120 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:44.961767 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:44.962520 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:44.962528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:44.963033 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:44.963713 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:44.963748 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.964834 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.964840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:44.966340 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:44.966355 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:44.967494 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:44.967503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:44.967789 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:44.968471 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:44.968491 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:44.970863 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:44.974877 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:44.974884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:44.974897 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:44.975322 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:44.975782 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:44.975785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:44.975787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:44.975985 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:44.976614 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:44.976620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:44.976962 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 22:53:44.977217 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 22:53:44.977597 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":254,"threads_started":1}
I20260807 22:53:44.980309 26945 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:53:44.980676 26945 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:44.981554 26945 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:53:44.983088 26600 ts_manager.cc:194] Registered new tserver with Master: e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409)
I20260807 22:53:44.983968 26600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:59541
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:53:45.028504 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:53:45.365671 26952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:53:45.365980 26952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:53:45.366153 26952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:53:45.373373 26952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:53:45.373520 26952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:53:45.373703 26952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:53:45.383379 26952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:53:45.384622 26952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:53:45.385912 26952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:53:45.398777 26960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:45.398835 26957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:53:45.398860 26958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:53:45.399698 26952 server_base.cc:1034] running on GCE node
I20260807 22:53:45.400291 26952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:53:45.401145 26952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:53:45.402343 26952 hybrid_clock.cc:648] HybridClock initialized: now 1786143225402302 us; error 32 us; skew 500 ppm
I20260807 22:53:45.402690 26952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:53:45 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143225, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:53:45.405725 26952 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:53:45.407083 26952 webserver.cc:460] Webserver started at http://127.9.194.195:40809/ using document root <none> and password file <none>
I20260807 22:53:45.407826 26952 fs_manager.cc:359] Metadata directory not provided
I20260807 22:53:45.407968 26952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:53:45.408279 26952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:53:45.411036 26952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "020629862db64f65bf5c1d4323c4ab47"
format_stamp: "Formatted at 2026-08-07 22:53:45 on dist-test-slave-5k54"
server_key: "cf7414f486ecb1d3c767f42fdc3f8cb5"
server_key_iv: "9c01b9f8b59312c0d7f0e2abd42136c1"
server_key_version: "encryptionkey@0"
I20260807 22:53:45.411830 26952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "020629862db64f65bf5c1d4323c4ab47"
format_stamp: "Formatted at 2026-08-07 22:53:45 on dist-test-slave-5k54"
server_key: "cf7414f486ecb1d3c767f42fdc3f8cb5"
server_key_iv: "9c01b9f8b59312c0d7f0e2abd42136c1"
server_key_version: "encryptionkey@0"
I20260807 22:53:45.416867 26952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:53:45.420254 26968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:45.421594 26952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:53:45.421784 26952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "020629862db64f65bf5c1d4323c4ab47"
format_stamp: "Formatted at 2026-08-07 22:53:45 on dist-test-slave-5k54"
server_key: "cf7414f486ecb1d3c767f42fdc3f8cb5"
server_key_iv: "9c01b9f8b59312c0d7f0e2abd42136c1"
server_key_version: "encryptionkey@0"
I20260807 22:53:45.421984 26952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:53:45.440246 26952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:53:45.444203 26952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:53:45.444494 26952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:53:45.445446 26952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:53:45.446789 26952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:53:45.446893 26952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:45.447042 26952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:53:45.447114 26952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:53:45.465291 26801 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:53:45.481333 26952 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39107
I20260807 22:53:45.481493 27091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39107 every 8 connection(s)
I20260807 22:53:45.482625 26952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:53:45.484997 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 26952
I20260807 22:53:45.485231 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:53:45.485692 9995 external_mini_cluster.cc:1428] Setting key e55e3edeacc69bf9ed4dde05f615a69f
Aug 07 22:53:45 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143225, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:53:45.502758 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.485155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:45175 (local address 127.9.194.254:45469)
0807 22:53:45.487688 (+ 2533us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.487694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.487717 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:53:45.489436 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.489441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.489504 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.489595 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:45.491003 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.491832 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.493263 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.493690 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.493799 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.497119 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:45.497123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:45.497126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:45.497158 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:45.499770 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.500652 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.500657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.500659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.500825 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.501328 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.501331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.501333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.501609 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:45.501723 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.502324 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.502493 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2403,"thread_start_us":2290,"threads_started":1}
I20260807 22:53:45.503755 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.485796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:45175)
0807 22:53:45.486794 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.486852 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.489184 (+ 2332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.489794 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.489802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.490295 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:53:45.490864 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.490882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.491970 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.491974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.493148 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.493159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.493887 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.493891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.494090 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.494759 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:53:45.494780 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:53:45.496925 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:53:45.499893 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:45.499917 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:45.499948 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:45.500520 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:45.500947 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:53:45.500952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:53:45.500956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:53:45.501194 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:53:45.501716 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:53:45.501721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:53:45.502023 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.502241 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.502598 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":396,"threads_started":1}
I20260807 22:53:45.505106 27092 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:53:45.505398 27092 heartbeater.cc:464] Registering TS with master...
I20260807 22:53:45.506102 27092 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:53:45.507498 26600 ts_manager.cc:194] Registered new tserver with Master: 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
I20260807 22:53:45.508251 26600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:45175
I20260807 22:53:45.513499 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:53:45.531059 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.518026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.9.194.254:45469)
0807 22:53:45.518248 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.518257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.518702 (+ 445us) server_negotiation.cc:407] Connection header received
0807 22:53:45.518839 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.518842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.518912 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.518993 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:45.520492 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.521283 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.522873 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.523432 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.523581 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.526067 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:45.526073 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:45.526078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:45.526159 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:45.528777 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.529405 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.529408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.529411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.529534 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.529866 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.529869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.529871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.530161 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:45.530337 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.530558 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.530711 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:53:45.534752 26600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:53:45.541640 26600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:53:45.574921 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.555704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.9.194.193:39815)
0807 22:53:45.566900 (+ 11196us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.566917 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.567466 (+ 549us) server_negotiation.cc:407] Connection header received
0807 22:53:45.567687 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.567697 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.567921 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.568136 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:45.568853 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.570497 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.572596 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.573733 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.573884 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574142 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.574385 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.574556 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10946,"thread_start_us":10774,"threads_started":1}
I20260807 22:53:45.575299 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.554984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:41680)
0807 22:53:45.567321 (+ 12337us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.567335 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.567513 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.568157 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.568162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.568188 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:45.568644 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.568658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.570640 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.570645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.572383 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.572402 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.573947 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.573953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.574140 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574206 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.574341 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.574566 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12221,"mutex_wait_us":19,"thread_start_us":12080,"threads_started":1}
I20260807 22:53:45.575471 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.554547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.9.194.195:39107)
0807 22:53:45.565784 (+ 11237us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.565795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.566162 (+ 367us) server_negotiation.cc:407] Connection header received
0807 22:53:45.566839 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.566845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.567065 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.567224 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:45.568280 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.569974 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.572688 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.574015 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.574100 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574315 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.574945 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.575106 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10992,"thread_start_us":11241,"threads_started":1}
I20260807 22:53:45.575456 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.553514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41409 (local address 127.0.0.1:41598)
0807 22:53:45.559199 (+ 5685us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.559268 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.559481 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.567173 (+ 7692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.567182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.567221 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:45.567758 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.567776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.569708 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.569716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.572268 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.572285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.574453 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.574461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.574726 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574762 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.574840 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.574996 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":34,"thread_start_us":466,"threads_started":1}
I20260807 22:53:45.576193 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.554103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39107 (local address 127.0.0.1:37206)
0807 22:53:45.566040 (+ 11937us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.566063 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.566643 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.567562 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.567568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.567599 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:45.568037 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.568048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.570106 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.570111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.572502 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.572534 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.574514 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.574520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.574740 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574765 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.575713 (+ 948us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.575837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11817,"mutex_wait_us":118,"thread_start_us":12034,"threads_started":1}
I20260807 22:53:45.576203 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.553780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.9.194.194:41409)
0807 22:53:45.566429 (+ 12649us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.566438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.566464 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:53:45.566643 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.566652 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.566913 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.567768 (+ 855us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:45.568102 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.569530 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.572465 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.573956 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.574099 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.574288 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.575704 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.575871 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12465,"thread_start_us":12308,"threads_started":1}
I20260807 22:53:45.579483 26730 tablet_service.cc:1467] Processing CreateTablet for tablet 80e2cabcf4474558b45a30bc023a1667 (DEFAULT_TABLE table=test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:45.579500 26874 tablet_service.cc:1467] Processing CreateTablet for tablet 80e2cabcf4474558b45a30bc023a1667 (DEFAULT_TABLE table=test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:45.580880 26730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e2cabcf4474558b45a30bc023a1667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:45.580947 26874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e2cabcf4474558b45a30bc023a1667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:45.582196 27022 tablet_service.cc:1467] Processing CreateTablet for tablet 80e2cabcf4474558b45a30bc023a1667 (DEFAULT_TABLE table=test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:53:45.583556 27022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e2cabcf4474558b45a30bc023a1667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:53:45.591813 27113 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Bootstrap starting.
I20260807 22:53:45.595995 27113 tablet_bootstrap.cc:654] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:45.596997 27113 log.cc:824] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:45.599615 27116 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Bootstrap starting.
I20260807 22:53:45.599851 27113 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: No bootstrap required, opened a new log
I20260807 22:53:45.599965 27114 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Bootstrap starting.
I20260807 22:53:45.600129 27113 ts_tablet_manager.cc:1397] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:53:45.603143 27116 tablet_bootstrap.cc:654] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:45.603647 27114 tablet_bootstrap.cc:654] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Neither blocks nor log segments found. Creating new log.
I20260807 22:53:45.604461 27116 log.cc:824] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:45.604879 27114 log.cc:824] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:53:45.607309 27116 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: No bootstrap required, opened a new log
I20260807 22:53:45.607591 27116 ts_tablet_manager.cc:1397] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260807 22:53:45.607911 27114 tablet_bootstrap.cc:492] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: No bootstrap required, opened a new log
I20260807 22:53:45.608178 27114 ts_tablet_manager.cc:1397] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 22:53:45.608757 27113 raft_consensus.cc:348] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.609167 27113 raft_consensus.cc:374] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:45.609273 27113 raft_consensus.cc:729] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb23ff56e0646ef83852f3f7a62ca92, State: Initialized, Role: FOLLOWER
I20260807 22:53:45.609880 27113 consensus_queue.cc:260] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.611764 27113 ts_tablet_manager.cc:1428] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 22:53:45.616858 27114 raft_consensus.cc:348] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.617218 27114 raft_consensus.cc:374] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:45.617323 27114 raft_consensus.cc:729] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e266ab6347834bcf8379e819178980c1, State: Initialized, Role: FOLLOWER
I20260807 22:53:45.617527 27116 raft_consensus.cc:348] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.617859 27116 raft_consensus.cc:374] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:53:45.617982 27116 raft_consensus.cc:729] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020629862db64f65bf5c1d4323c4ab47, State: Initialized, Role: FOLLOWER
I20260807 22:53:45.617980 27114 consensus_queue.cc:260] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.618559 27116 consensus_queue.cc:260] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.619160 26945 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:53:45.619683 27092 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:53:45.619849 27114 ts_tablet_manager.cc:1428] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260807 22:53:45.620252 27116 ts_tablet_manager.cc:1428] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260807 22:53:45.668262 27119 raft_consensus.cc:482] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:53:45.668536 27119 raft_consensus.cc:504] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.670256 27119 leader_election.cc:288] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
I20260807 22:53:45.677055 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.671260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:41053 (local address 127.9.194.194:41409)
0807 22:53:45.671530 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.671538 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.671785 (+ 247us) server_negotiation.cc:407] Connection header received
0807 22:53:45.672030 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.672035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.672094 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.672190 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:45.672716 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.673393 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.675272 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.676100 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.676277 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.676356 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.676662 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.676787 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:53:45.677160 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.670822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:48473 (local address 127.9.194.195:39107)
0807 22:53:45.671066 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.671076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.672138 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 22:53:45.672323 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.672332 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.672389 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.672486 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:53:45.673027 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.673721 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.675487 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.676244 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.676428 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.676504 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.676777 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.676904 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:53:45.677201 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.670427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41409 (local address 127.9.194.193:41053)
0807 22:53:45.671662 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.671686 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.671865 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.672213 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.672217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.672242 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:45.672569 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.672577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.673536 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.673543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.675108 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.675120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.676308 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.676314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.676482 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.676530 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.676637 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.676761 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"mutex_wait_us":99,"thread_start_us":903,"threads_started":1}
I20260807 22:53:45.677337 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.670712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39107 (local address 127.9.194.193:48473)
0807 22:53:45.672034 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0807 22:53:45.672048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:53:45.672182 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:53:45.672507 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:53:45.672511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:53:45.672541 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:53:45.672880 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.672888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.673856 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.673861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.675324 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:53:45.675336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.676475 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.676481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:53:45.676632 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.676655 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:53:45.676753 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:53:45.676878 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"thread_start_us":1010,"threads_started":1}
I20260807 22:53:45.678279 26897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e2cabcf4474558b45a30bc023a1667" candidate_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e266ab6347834bcf8379e819178980c1" is_pre_election: true
I20260807 22:53:45.678314 27042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e2cabcf4474558b45a30bc023a1667" candidate_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "020629862db64f65bf5c1d4323c4ab47" is_pre_election: true
I20260807 22:53:45.678906 27042 raft_consensus.cc:2454] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb23ff56e0646ef83852f3f7a62ca92 in term 0.
I20260807 22:53:45.678907 26897 raft_consensus.cc:2454] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb23ff56e0646ef83852f3f7a62ca92 in term 0.
I20260807 22:53:45.679656 26681 leader_election.cc:302] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92, e266ab6347834bcf8379e819178980c1; no voters:
I20260807 22:53:45.680078 27119 raft_consensus.cc:2781] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:53:45.680197 27119 raft_consensus.cc:482] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:53:45.680307 27119 raft_consensus.cc:3037] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:45.682713 27119 raft_consensus.cc:504] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.683305 27119 leader_election.cc:288] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 1 election: Requested vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
I20260807 22:53:45.683734 26897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e2cabcf4474558b45a30bc023a1667" candidate_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e266ab6347834bcf8379e819178980c1"
I20260807 22:53:45.683828 27042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e2cabcf4474558b45a30bc023a1667" candidate_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "020629862db64f65bf5c1d4323c4ab47"
I20260807 22:53:45.683979 26897 raft_consensus.cc:3037] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:53:45.684005 27042 raft_consensus.cc:3037] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:53:45.684314 26802 tablet.cc:2369] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:45.687211 26897 raft_consensus.cc:2454] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb23ff56e0646ef83852f3f7a62ca92 in term 1.
I20260807 22:53:45.687211 27042 raft_consensus.cc:2454] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb23ff56e0646ef83852f3f7a62ca92 in term 1.
I20260807 22:53:45.687685 26684 leader_election.cc:302] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 020629862db64f65bf5c1d4323c4ab47, 1cb23ff56e0646ef83852f3f7a62ca92, e266ab6347834bcf8379e819178980c1; no voters:
I20260807 22:53:45.688035 27119 raft_consensus.cc:2781] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:53:45.688469 27119 raft_consensus.cc:686] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 LEADER]: Becoming Leader. State: Replica: 1cb23ff56e0646ef83852f3f7a62ca92, State: Running, Role: LEADER
I20260807 22:53:45.688969 27119 consensus_queue.cc:237] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } }
I20260807 22:53:45.693329 26599 catalog_manager.cc:5495] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } health_report { overall_health: UNKNOWN } } }
W20260807 22:53:45.708313 26946 tablet.cc:2369] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:53:45.734205 27093 tablet.cc:2369] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:53:45.736487 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.731194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.9.194.193:39815)
0807 22:53:45.731386 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.731390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.731406 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:53:45.731544 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.731548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.731637 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.731757 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:53:45.732076 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.732776 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.734247 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.734745 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.734886 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.735379 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:53:45.735884 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:53:45.736080 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.736140 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.736228 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:53:45.745074 27042 raft_consensus.cc:1261] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Refusing update from remote peer 1cb23ff56e0646ef83852f3f7a62ca92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:45.745071 26897 raft_consensus.cc:1261] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Refusing update from remote peer 1cb23ff56e0646ef83852f3f7a62ca92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:53:45.746125 27124 consensus_queue.cc:1035] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:45.746455 27119 consensus_queue.cc:1035] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:53:45.774422 26599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:45.774741 26599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:53:45.778163 26599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Sending DeleteTablet for 3 replicas of tablet 80e2cabcf4474558b45a30bc023a1667
I20260807 22:53:45.779213 26874 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e2cabcf4474558b45a30bc023a1667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:45 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41598
I20260807 22:53:45.779354 26730 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e2cabcf4474558b45a30bc023a1667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:45 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:41680
I20260807 22:53:45.779531 27022 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e2cabcf4474558b45a30bc023a1667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:53:45 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37206
I20260807 22:53:45.780169 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26665
I20260807 22:53:45.790661 27139 tablet_replica.cc:330] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: stopping tablet replica
I20260807 22:53:45.791217 27139 raft_consensus.cc:2229] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:45.791869 27139 raft_consensus.cc:2258] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:45.792543 27137 tablet_replica.cc:330] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: stopping tablet replica
I20260807 22:53:45.792954 27137 raft_consensus.cc:2229] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:53:45.793437 27137 raft_consensus.cc:2258] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:45.794394 27139 ts_tablet_manager.cc:1905] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:45.795545 27137 ts_tablet_manager.cc:1905] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:45.801208 27139 ts_tablet_manager.cc:1918] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:45.801378 27139 log.cc:1196] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/80e2cabcf4474558b45a30bc023a1667
I20260807 22:53:45.801875 27139 ts_tablet_manager.cc:1939] T 80e2cabcf4474558b45a30bc023a1667 P 020629862db64f65bf5c1d4323c4ab47: Deleting consensus metadata
I20260807 22:53:45.802517 27137 ts_tablet_manager.cc:1918] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:45.802673 27137 log.cc:1196] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/80e2cabcf4474558b45a30bc023a1667
I20260807 22:53:45.803198 27137 ts_tablet_manager.cc:1939] T 80e2cabcf4474558b45a30bc023a1667 P e266ab6347834bcf8379e819178980c1: Deleting consensus metadata
I20260807 22:53:45.803959 27138 tablet_replica.cc:330] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: stopping tablet replica
I20260807 22:53:45.804394 27138 raft_consensus.cc:2229] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:53:45.805079 26584 catalog_manager.cc:4841] TS e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): tablet 80e2cabcf4474558b45a30bc023a1667 (table test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]) successfully deleted
I20260807 22:53:45.805148 27138 raft_consensus.cc:2258] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:53:45.806792 27138 ts_tablet_manager.cc:1905] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:53:45.809042 26587 catalog_manager.cc:4841] TS 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): tablet 80e2cabcf4474558b45a30bc023a1667 (table test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]) successfully deleted
I20260807 22:53:45.813259 27138 ts_tablet_manager.cc:1918] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:53:45.813452 27138 log.cc:1196] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/80e2cabcf4474558b45a30bc023a1667
I20260807 22:53:45.814060 27138 ts_tablet_manager.cc:1939] T 80e2cabcf4474558b45a30bc023a1667 P 1cb23ff56e0646ef83852f3f7a62ca92: Deleting consensus metadata
I20260807 22:53:45.816344 26584 catalog_manager.cc:4841] TS 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815): tablet 80e2cabcf4474558b45a30bc023a1667 (table test-table [id=6f7210db08d740ffbe93ecaeb172fa7e]) successfully deleted
Aug 07 22:53:45 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:53:45.826114 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.781503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.9.194.193:39815)
0807 22:53:45.781676 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.781682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.804243 (+ 22561us) server_negotiation.cc:407] Connection header received
0807 22:53:45.804560 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.804569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.804658 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.804733 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:45.807499 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.808696 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.811172 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.814062 (+ 2890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.814552 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.819731 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:45.819743 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:45.819756 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:45.819792 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:45.822858 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.823611 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.823618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.823620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.823770 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.824353 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.824358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.824361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.824741 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:45.825124 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.825383 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.825656 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:53:45.826839 26796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:45.926987 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26665
I20260807 22:53:45.947728 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26807
Aug 07 22:53:45 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:53:45.964151 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:45.948493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.9.194.194:41409)
0807 22:53:45.948662 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:45.948666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:45.948684 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:53:45.948814 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:45.948817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:45.948888 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:45.948983 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:45.950313 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.951063 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.952559 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:45.952941 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:45.953039 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:45.957612 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:45.957623 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:45.957640 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:45.957682 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:45.961490 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.962106 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.962115 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.962119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.962252 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:45.962582 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:45.962587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:45.962590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:45.963048 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:45.963314 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:45.963569 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 22:53:45.963794 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:53:45.964900 26940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:46.054868 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26807
I20260807 22:53:46.076480 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26952
Aug 07 22:53:46 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:53:46.092746 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:53:46.077372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.9.194.195:39107)
0807 22:53:46.077552 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:53:46.077559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:53:46.077586 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:53:46.077770 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:53:46.077775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:53:46.077845 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:53:46.077937 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:53:46.079497 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:46.080234 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:46.081898 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:53:46.082288 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:53:46.082442 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:53:46.087009 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:53:46.087025 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:53:46.087040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:53:46.087082 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:53:46.089977 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:46.090652 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:46.090660 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:46.090662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:46.090826 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:53:46.091172 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:53:46.091175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:53:46.091177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:53:46.091688 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:53:46.092004 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 22:53:46.092201 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 22:53:46.092375 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:53:46.093508 27087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:46.182792 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26952
I20260807 22:53:46.202459 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 26567
I20260807 22:53:46.203735 26630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:53:46.315603 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 26567
I20260807 22:54:06.341022 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--webserver_interface=127.9.194.254
--webserver_port=45985
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:06.675889 27146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:06.676237 27146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:06.676396 27146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:06.684044 27146 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:54:06.684186 27146 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:54:06.684281 27146 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:54:06.684383 27146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:06.684486 27146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:06.684593 27146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:06.684681 27146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:06.684762 27146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:06.695089 27146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=45985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:06.696497 27146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:06.697888 27146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:06.724167 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:06.724268 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:06.724200 27156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:06.729077 27146 server_base.cc:1034] running on GCE node
I20260807 22:54:06.730003 27146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:06.731683 27146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:06.732913 27146 hybrid_clock.cc:648] HybridClock initialized: now 1786143246732843 us; error 37 us; skew 500 ppm
I20260807 22:54:06.733333 27146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:06 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:06.737305 27146 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:54:06.739066 27146 webserver.cc:460] Webserver started at http://127.9.194.254:45985/ using document root <none> and password file <none>
I20260807 22:54:06.739964 27146 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:06.740106 27146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:06.746254 27146 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:54:06.749284 27164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:06.750592 27146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:54:06.750792 27146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a9e67bee9fe488ba2c5b74915883421"
format_stamp: "Formatted at 2026-08-07 22:53:43 on dist-test-slave-5k54"
server_key: "2e78e037bb636923cd4e2a53a2f4a942"
server_key_iv: "44bc0df3f2dd74c3f0b60cce97df1881"
server_key_version: "encryptionkey@0"
I20260807 22:54:06.751895 27146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:06.767882 27146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:06.771993 27146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:06.772311 27146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:06.792125 27146 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:45469
I20260807 22:54:06.792173 27215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:45469 every 8 connection(s)
I20260807 22:54:06.793700 27146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:06.800593 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27146
I20260807 22:54:06.801180 27216 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:54:06.801726 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:39815
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=37145
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:54:06.806602 27216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap starting.
I20260807 22:54:06.829536 27216 log.cc:824] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:06.842147 27216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:54:06.842734 27216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap complete.
I20260807 22:54:06.852216 27216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:06.853080 27216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Initialized, Role: FOLLOWER
I20260807 22:54:06.853732 27216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:06.853971 27216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:06.854105 27216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:06.854302 27216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:54:06.858479 27216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:06.859035 27216 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e67bee9fe488ba2c5b74915883421; no voters:
I20260807 22:54:06.859699 27216 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:54:06.859843 27220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:54:06.861104 27220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 LEADER]: Becoming Leader. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Running, Role: LEADER
I20260807 22:54:06.861815 27220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:06.862218 27216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:06.864771 27221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:54:06.864760 27223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e67bee9fe488ba2c5b74915883421. Latest consensus state: current_term: 2 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:54:06.865315 27221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:06.865350 27223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:06.866856 27227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:06.888288 27227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:06.889918 27227 catalog_manager.cc:1259] Loaded cluster ID: f0b91b432696475181e3a406c408ada9
I20260807 22:54:06.890057 27227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:06.895556 27227 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:06.897163 27227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Loaded TSK: 0
I20260807 22:54:06.919373 27239 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:54:06.930738 27227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Generated new TSK 1
I20260807 22:54:06.931137 27227 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:54:07.164959 27218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:07.165234 27218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:07.165390 27218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:07.172639 27218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:07.172744 27218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:07.172910 27218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:07.182763 27218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:39815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=37145
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:07.184106 27218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:07.185437 27218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:07.199013 27246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:07.198966 27247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:07.199714 27249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:07.200141 27218 server_base.cc:1034] running on GCE node
I20260807 22:54:07.200896 27218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:07.202142 27218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:07.203392 27218 hybrid_clock.cc:648] HybridClock initialized: now 1786143247203317 us; error 59 us; skew 500 ppm
I20260807 22:54:07.203779 27218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:07 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143247, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:07.207769 27218 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:54:07.209331 27218 webserver.cc:460] Webserver started at http://127.9.194.193:37145/ using document root <none> and password file <none>
I20260807 22:54:07.210006 27218 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:07.210150 27218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:07.215937 27218 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:54:07.218883 27257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.220379 27218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:07.220568 27218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "bf9364736a479a9cecd85d694143361e"
server_key_iv: "2bd5e32148c48f5cdf5381371377058a"
server_key_version: "encryptionkey@0"
I20260807 22:54:07.221534 27218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:07.235343 27218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:07.239770 27218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:07.240065 27218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:07.241111 27218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:07.242486 27218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:07.242621 27218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.242741 27218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:07.242833 27218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.278486 27218 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39815
I20260807 22:54:07.278579 27374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39815 every 8 connection(s)
I20260807 22:54:07.280073 27218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:54:07.284138 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27218
I20260807 22:54:07.284888 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:41409
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=36453
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:54:07 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143247, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:07.304368 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:07.282873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:48173 (local address 127.9.194.254:45469)
0807 22:54:07.283475 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:07.283520 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:07.284131 (+ 611us) server_negotiation.cc:407] Connection header received
0807 22:54:07.286445 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:07.286458 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:07.286802 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:07.287168 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:07.288828 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.290263 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.291895 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.292544 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.292739 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:07.296478 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:07.296490 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:07.296510 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:07.296563 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:07.299869 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:07.300823 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:07.300829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:07.300835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:07.300991 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:07.301495 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:07.301498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:07.301500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:07.301933 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:07.302073 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:07.303081 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0807 22:54:07.303387 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":249,"threads_started":1}
I20260807 22:54:07.304895 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:07.283145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:48173)
0807 22:54:07.283957 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:07.283993 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:07.285877 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:07.287321 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:07.287334 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:07.287991 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:07.288578 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:07.288595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.290431 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.290438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:07.291740 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:07.291756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.292785 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.292793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:07.293061 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:07.293761 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:07.293783 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:07.296166 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:07.300020 (+ 3854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:07.300031 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:07.300048 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:07.300689 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:07.301114 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:07.301121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:07.301128 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:07.301378 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:07.302095 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:07.302105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:07.302544 (+ 439us) client_negotiation.cc:769] Sending connection context
0807 22:54:07.302898 (+ 354us) client_negotiation.cc:240] Negotiation successful
0807 22:54:07.303500 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":377,"threads_started":1}
I20260807 22:54:07.307639 27375 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:54:07.307938 27375 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:07.308645 27375 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:54:07.310858 27181 ts_manager.cc:194] Registered new tserver with Master: 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815)
I20260807 22:54:07.312372 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:48173
W20260807 22:54:07.641733 27380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:07.642056 27380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:07.642233 27380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:07.649714 27380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:07.649940 27380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:07.650141 27380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:54:07.660152 27380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:41409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=36453
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:07.661469 27380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:07.662748 27380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:07.675884 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:07.675933 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:07.676395 27389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:07.676578 27380 server_base.cc:1034] running on GCE node
I20260807 22:54:07.677395 27380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:07.678398 27380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:07.679642 27380 hybrid_clock.cc:648] HybridClock initialized: now 1786143247679602 us; error 36 us; skew 500 ppm
I20260807 22:54:07.679965 27380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:07 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143247, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:07.683027 27380 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:54:07.684427 27380 webserver.cc:460] Webserver started at http://127.9.194.194:36453/ using document root <none> and password file <none>
I20260807 22:54:07.685120 27380 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:07.685278 27380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:07.690860 27380 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 22:54:07.693548 27396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.694826 27380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:07.695003 27380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "e266ab6347834bcf8379e819178980c1"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "ef7a69c02918e09696525b99ae7997fc"
server_key_iv: "80f7d9136ca2b9016e2cdee79ae29bae"
server_key_version: "encryptionkey@0"
I20260807 22:54:07.696012 27380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:07.714993 27380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:07.718923 27380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:07.719201 27380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:07.720290 27380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:07.721535 27380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:07.721619 27380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.721699 27380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:07.721752 27380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:07.754920 27380 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:41409
I20260807 22:54:07.755089 27514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:41409 every 8 connection(s)
I20260807 22:54:07.756618 27380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:54:07.757634 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27380
I20260807 22:54:07.758445 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:39107
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=40809
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:54:07 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143247, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:07.779979 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:07.759629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:46565 (local address 127.9.194.254:45469)
0807 22:54:07.759872 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:07.759882 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:07.761148 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 22:54:07.763998 (+ 2850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:07.764002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:07.764074 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:07.764162 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:07.766361 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.767046 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.769561 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.770015 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.770167 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:07.773710 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:07.773717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:07.773724 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:07.773769 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:07.776882 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:07.777630 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:07.777636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:07.777639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:07.777770 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:07.778218 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:07.778223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:07.778228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:07.778510 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:07.778677 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:07.779371 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 22:54:07.779626 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:54:07.780972 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:07.760131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:46565)
0807 22:54:07.760966 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:07.761016 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:07.763679 (+ 2663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:07.764455 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:07.764469 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:07.765332 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:07.766176 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:07.766204 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.767223 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.767232 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:07.769399 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:07.769416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:07.770242 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:07.770248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:07.770455 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:07.771177 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:07.771203 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:07.773418 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:07.777024 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:07.777034 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:07.777051 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:07.777485 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:07.777885 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:07.777891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:07.777894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:07.778095 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:07.778652 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:07.778660 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:07.779002 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 22:54:07.779269 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 22:54:07.779832 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":311,"threads_started":1}
I20260807 22:54:07.782697 27515 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:54:07.782998 27515 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:07.783764 27515 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:54:07.785405 27181 ts_manager.cc:194] Registered new tserver with Master: e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409)
I20260807 22:54:07.786909 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:46565
W20260807 22:54:08.104910 27518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:08.105189 27518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:08.105284 27518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:08.112545 27518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:08.112651 27518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:08.112761 27518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:54:08.122452 27518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:39107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=40809
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:08.123742 27518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:08.125090 27518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:08.138032 27527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:08.138106 27524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:08.138182 27523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:08.139014 27518 server_base.cc:1034] running on GCE node
I20260807 22:54:08.139704 27518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:08.140604 27518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:08.141831 27518 hybrid_clock.cc:648] HybridClock initialized: now 1786143248141786 us; error 59 us; skew 500 ppm
I20260807 22:54:08.142158 27518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:08 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143248, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:08.148610 27518 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:54:08.150142 27518 webserver.cc:460] Webserver started at http://127.9.194.195:40809/ using document root <none> and password file <none>
I20260807 22:54:08.150813 27518 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:08.150981 27518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:08.156322 27518 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 22:54:08.158993 27534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:08.160323 27518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:54:08.160507 27518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "020629862db64f65bf5c1d4323c4ab47"
format_stamp: "Formatted at 2026-08-07 22:53:45 on dist-test-slave-5k54"
server_key: "cf7414f486ecb1d3c767f42fdc3f8cb5"
server_key_iv: "9c01b9f8b59312c0d7f0e2abd42136c1"
server_key_version: "encryptionkey@0"
I20260807 22:54:08.161496 27518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:08.194818 27518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:08.198786 27518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:08.199064 27518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:08.200273 27518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:08.201630 27518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:08.201741 27518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:08.201875 27518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:08.201977 27518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:08.235747 27518 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39107
I20260807 22:54:08.235841 27655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39107 every 8 connection(s)
I20260807 22:54:08.237243 27518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:54:08.240844 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27518
Aug 07 22:54:08 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143248, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:08.257277 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.241792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.9.194.254:45469)
0807 22:54:08.242230 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.242236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.242626 (+ 390us) server_negotiation.cc:407] Connection header received
0807 22:54:08.242776 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.242782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.242863 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.242986 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.244955 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.245951 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.247482 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.248171 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.248311 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.251104 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.251108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.251111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.251140 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.254179 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.254875 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.254889 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.254893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.255038 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.255487 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.255492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.255497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.255900 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.256161 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.256584 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.256805 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"spinlock_wait_cycles":7552,"thread_start_us":220,"threads_started":1}
I20260807 22:54:08.259028 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.239829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:43975 (local address 127.9.194.254:45469)
0807 22:54:08.240046 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.240050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.241387 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 22:54:08.243741 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.243747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.243826 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.243934 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.245555 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.246468 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.248338 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.248972 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.249117 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.252671 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.252676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.252679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.252719 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.255687 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.256511 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.256516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.256520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.256637 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.257169 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.257174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.257177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.257541 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.257693 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.258463 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.258686 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:54:08.260046 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.240296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:43975)
0807 22:54:08.241216 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.241257 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.243408 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.244142 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.244149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.244640 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:08.245372 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.245407 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.246640 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.246644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.248197 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.248212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.249227 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.249233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.249436 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.250029 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:08.250050 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:08.252476 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:08.255806 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:08.255815 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:08.255829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:08.256402 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:08.256797 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:08.256802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:08.256805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:08.257026 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:08.257676 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:08.257683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:08.258069 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.258379 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.258891 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":335,"threads_started":1}
I20260807 22:54:08.261309 27656 heartbeater.cc:347] Connected to a master server at 127.9.194.254:45469
I20260807 22:54:08.261607 27656 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:08.262293 27656 heartbeater.cc:511] Master 127.9.194.254:45469 requested a full tablet report, sending...
I20260807 22:54:08.263742 27181 ts_manager.cc:194] Registered new tserver with Master: 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
I20260807 22:54:08.264581 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:43975
I20260807 22:54:08.272356 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:54:08.279951 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.273299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.9.194.254:45469)
0807 22:54:08.273520 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.273527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.273929 (+ 402us) server_negotiation.cc:407] Connection header received
0807 22:54:08.274192 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.274195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.274325 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.274481 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:54:08.274943 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.275965 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.277850 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.278356 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.278496 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.279023 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:54:08.279281 (+ 258us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 22:54:08.279702 (+ 421us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33420: token expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 22:54:08.280040 27378 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33420: token expired
W20260807 22:54:08.280647 27096 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.194.254:45469)
I20260807 22:54:08.293733 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.281170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33428 (local address 127.9.194.254:45469)
0807 22:54:08.281313 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.281320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.281718 (+ 398us) server_negotiation.cc:407] Connection header received
0807 22:54:08.281761 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.281764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.281827 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.281972 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.283282 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.284023 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.285306 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.285703 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.285832 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.288537 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.288542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.288546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.288575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.291101 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.291794 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.291801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.291804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.291936 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.292328 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.292333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.292336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.292647 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.292902 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.293219 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.293440 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:54:08.296828 27181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:54:08.304216 27181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:54:08.315452 27375 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:54:08.330260 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.316026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39107 (local address 127.0.0.1:52368)
0807 22:54:08.317336 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.317353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.317452 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.319132 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.319143 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.319184 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:08.319762 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.319778 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.322448 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.322458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.326283 (+ 3825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.326301 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.328864 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.328875 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.329212 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.329291 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.329427 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.329691 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"thread_start_us":1052,"threads_started":1}
I20260807 22:54:08.331996 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.316702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.9.194.195:39107)
0807 22:54:08.318358 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.318368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.318404 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:54:08.318506 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.318518 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.318816 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.320045 (+ 1229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:08.320388 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.322242 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.326523 (+ 4281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.328548 (+ 2025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.330814 (+ 2266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.331156 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.331245 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.331481 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1477,"thread_start_us":967,"threads_started":1}
I20260807 22:54:08.334885 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.315698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41409 (local address 127.0.0.1:36274)
0807 22:54:08.316521 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.316557 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.316722 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.325006 (+ 8284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.325014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.325050 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:08.325471 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.325481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.327939 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.327948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.331122 (+ 3174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.331135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.334139 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.334146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.334323 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.334355 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.334443 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.334577 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":426,"threads_started":1}
I20260807 22:54:08.334954 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.315925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.9.194.194:41409)
0807 22:54:08.324391 (+ 8466us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.324399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.324421 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:54:08.324490 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.324522 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.324764 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.325003 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:08.325677 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.327732 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.331272 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.332846 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.333004 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.333162 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.334518 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.334679 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8327,"thread_start_us":8216,"threads_started":1}
I20260807 22:54:08.336475 27588 tablet_service.cc:1467] Processing CreateTablet for tablet 7c45a5029048454fa28952a02927079c (DEFAULT_TABLE table=test-table [id=72fa6c81fe704e1cb4a66e04d7b2ad46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:08.337839 27588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c45a5029048454fa28952a02927079c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:08.338189 27446 tablet_service.cc:1467] Processing CreateTablet for tablet 7c45a5029048454fa28952a02927079c (DEFAULT_TABLE table=test-table [id=72fa6c81fe704e1cb4a66e04d7b2ad46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:08.339576 27446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c45a5029048454fa28952a02927079c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:08.341691 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.317958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38688)
0807 22:54:08.333418 (+ 15460us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.333433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.333568 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.335123 (+ 1555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.335128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.335154 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:08.335316 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.335337 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.337269 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.337275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.339043 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.339073 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.340919 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.340925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.341157 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.341189 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.341289 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.341433 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15366,"thread_start_us":15724,"threads_started":1}
I20260807 22:54:08.341857 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.318379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.9.194.193:39815)
0807 22:54:08.330180 (+ 11801us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.330188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.333529 (+ 3341us) server_negotiation.cc:407] Connection header received
0807 22:54:08.334447 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.334455 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.334686 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.334866 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:08.335544 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.337095 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.339255 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.340729 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.340884 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.341127 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.341352 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.341512 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11609,"thread_start_us":11798,"threads_started":1}
I20260807 22:54:08.345607 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 7c45a5029048454fa28952a02927079c (DEFAULT_TABLE table=test-table [id=72fa6c81fe704e1cb4a66e04d7b2ad46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:08.346961 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c45a5029048454fa28952a02927079c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:08.352830 27674 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Bootstrap starting.
I20260807 22:54:08.355706 27674 tablet_bootstrap.cc:654] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:08.356971 27674 log.cc:824] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:08.358433 27675 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Bootstrap starting.
I20260807 22:54:08.360317 27674 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: No bootstrap required, opened a new log
I20260807 22:54:08.360610 27674 ts_tablet_manager.cc:1397] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:54:08.362679 27675 tablet_bootstrap.cc:654] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:08.362944 27677 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Bootstrap starting.
I20260807 22:54:08.364024 27675 log.cc:824] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:08.366431 27677 tablet_bootstrap.cc:654] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:08.367163 27675 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: No bootstrap required, opened a new log
I20260807 22:54:08.367472 27675 ts_tablet_manager.cc:1397] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 22:54:08.367508 27677 log.cc:824] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:08.369903 27677 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: No bootstrap required, opened a new log
I20260807 22:54:08.370162 27677 ts_tablet_manager.cc:1397] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260807 22:54:08.371085 27674 raft_consensus.cc:348] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.371562 27674 raft_consensus.cc:374] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:08.371680 27674 raft_consensus.cc:729] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e266ab6347834bcf8379e819178980c1, State: Initialized, Role: FOLLOWER
I20260807 22:54:08.372397 27674 consensus_queue.cc:260] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.373535 27515 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:54:08.374171 27674 ts_tablet_manager.cc:1428] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 22:54:08.378321 27677 raft_consensus.cc:348] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.378774 27677 raft_consensus.cc:374] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:08.378902 27677 raft_consensus.cc:729] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb23ff56e0646ef83852f3f7a62ca92, State: Initialized, Role: FOLLOWER
I20260807 22:54:08.379555 27677 consensus_queue.cc:260] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.380155 27675 raft_consensus.cc:348] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.380484 27675 raft_consensus.cc:374] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:08.380596 27675 raft_consensus.cc:729] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020629862db64f65bf5c1d4323c4ab47, State: Initialized, Role: FOLLOWER
I20260807 22:54:08.381352 27677 ts_tablet_manager.cc:1428] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260807 22:54:08.381175 27675 consensus_queue.cc:260] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.382285 27656 heartbeater.cc:503] Master 127.9.194.254:45469 was elected leader, sending a full tablet report...
I20260807 22:54:08.382853 27675 ts_tablet_manager.cc:1428] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260807 22:54:08.441917 27681 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:08.442284 27681 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.444039 27681 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107), 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815)
I20260807 22:54:08.451471 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.444717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:51453 (local address 127.9.194.193:39815)
0807 22:54:08.444974 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.444984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.445620 (+ 636us) server_negotiation.cc:407] Connection header received
0807 22:54:08.445915 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.445923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.446024 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.446122 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:08.446772 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.447675 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.449595 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.450386 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.450515 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.450608 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.451059 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.451190 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:54:08.451491 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.444998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:55283 (local address 127.9.194.195:39107)
0807 22:54:08.445212 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.445218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.446028 (+ 810us) server_negotiation.cc:407] Connection header received
0807 22:54:08.446190 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.446195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.446279 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.446395 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:08.447031 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.447910 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.449684 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.450490 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.450651 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.450733 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.451073 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.451193 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:54:08.451618 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.444457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39107 (local address 127.9.194.194:55283)
0807 22:54:08.445910 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.445927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.446050 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.446411 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.446414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.446434 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:08.446848 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.446862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.448057 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.448061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.449539 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.449555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.450717 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.450722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.450875 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.450922 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.451033 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.451165 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1288,"mutex_wait_us":25,"thread_start_us":715,"threads_started":1}
I20260807 22:54:08.452003 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.444340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.9.194.194:51453)
0807 22:54:08.445510 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.445535 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:08.445749 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:08.446166 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:08.446171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:08.446196 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:08.446615 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.446625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.447807 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.447813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.449425 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:08.449437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.450607 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.450613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:08.450821 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.450911 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 22:54:08.450985 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:54:08.451131 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":100,"thread_start_us":1224,"threads_started":1}
I20260807 22:54:08.452706 27611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c45a5029048454fa28952a02927079c" candidate_uuid: "e266ab6347834bcf8379e819178980c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "020629862db64f65bf5c1d4323c4ab47" is_pre_election: true
I20260807 22:54:08.453061 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c45a5029048454fa28952a02927079c" candidate_uuid: "e266ab6347834bcf8379e819178980c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" is_pre_election: true
I20260807 22:54:08.453195 27611 raft_consensus.cc:2454] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e266ab6347834bcf8379e819178980c1 in term 0.
I20260807 22:54:08.453543 27328 raft_consensus.cc:2454] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e266ab6347834bcf8379e819178980c1 in term 0.
I20260807 22:54:08.453903 27400 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1; no voters:
I20260807 22:54:08.454339 27681 raft_consensus.cc:2781] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:54:08.454478 27681 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:54:08.454608 27681 raft_consensus.cc:3037] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:08.457131 27681 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.457773 27681 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [CANDIDATE]: Term 1 election: Requested vote from peers 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107), 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815)
I20260807 22:54:08.458284 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c45a5029048454fa28952a02927079c" candidate_uuid: "e266ab6347834bcf8379e819178980c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
I20260807 22:54:08.458263 27611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c45a5029048454fa28952a02927079c" candidate_uuid: "e266ab6347834bcf8379e819178980c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "020629862db64f65bf5c1d4323c4ab47"
I20260807 22:54:08.458494 27328 raft_consensus.cc:3037] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:08.458494 27611 raft_consensus.cc:3037] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:08.461831 27328 raft_consensus.cc:2454] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e266ab6347834bcf8379e819178980c1 in term 1.
I20260807 22:54:08.461831 27611 raft_consensus.cc:2454] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e266ab6347834bcf8379e819178980c1 in term 1.
I20260807 22:54:08.462291 27400 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1; no voters:
I20260807 22:54:08.462582 27681 raft_consensus.cc:2781] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:08.463140 27681 raft_consensus.cc:686] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 1 LEADER]: Becoming Leader. State: Replica: e266ab6347834bcf8379e819178980c1, State: Running, Role: LEADER
I20260807 22:54:08.463696 27681 consensus_queue.cc:237] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:08.468469 27180 catalog_manager.cc:5495] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e266ab6347834bcf8379e819178980c1 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "e266ab6347834bcf8379e819178980c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20260807 22:54:08.487229 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.481740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.9.194.194:41409)
0807 22:54:08.481946 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.481951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.481969 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:54:08.482198 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.482202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.482276 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.482427 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:54:08.482844 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.483635 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.485103 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.485528 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.485666 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.486165 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:54:08.486649 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:54:08.486807 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.486865 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.486942 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 22:54:08.488916 27657 tablet.cc:2369] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:54:08.495694 27611 raft_consensus.cc:1261] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Refusing update from remote peer e266ab6347834bcf8379e819178980c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:08.495780 27328 raft_consensus.cc:1261] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Refusing update from remote peer e266ab6347834bcf8379e819178980c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:08.496711 27686 consensus_queue.cc:1035] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:08.497036 27681 consensus_queue.cc:1035] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:08.508159 27689 mvcc.cc:205] Tried to move back new op lower bound from 7316042745828302848 to 7316042745711415296. Current Snapshot: MvccSnapshot[applied={T|T < 7316042745828302848}]
I20260807 22:54:08.523223 27180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:08.523535 27180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:08.527071 27180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Sending DeleteTablet for 3 replicas of tablet 7c45a5029048454fa28952a02927079c
I20260807 22:54:08.528355 27588 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c45a5029048454fa28952a02927079c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:52368
I20260807 22:54:08.528380 27446 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c45a5029048454fa28952a02927079c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:36274
I20260807 22:54:08.528648 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c45a5029048454fa28952a02927079c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:08 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:38688
I20260807 22:54:08.529963 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27218
I20260807 22:54:08.540930 27701 tablet_replica.cc:330] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: stopping tablet replica
I20260807 22:54:08.541555 27701 raft_consensus.cc:2229] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:54:08.542516 27701 raft_consensus.cc:2258] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:08.543365 27700 tablet_replica.cc:330] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: stopping tablet replica
I20260807 22:54:08.543910 27700 raft_consensus.cc:2229] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:08.544510 27700 raft_consensus.cc:2258] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:08.546775 27700 ts_tablet_manager.cc:1905] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:08.553999 27700 ts_tablet_manager.cc:1918] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:08.554247 27700 log.cc:1196] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/7c45a5029048454fa28952a02927079c
I20260807 22:54:08.554900 27700 ts_tablet_manager.cc:1939] T 7c45a5029048454fa28952a02927079c P 020629862db64f65bf5c1d4323c4ab47: Deleting consensus metadata
I20260807 22:54:08.556349 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.531754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.9.194.193:39815)
0807 22:54:08.531975 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.531983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.532004 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:54:08.532073 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.532081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.532188 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.532308 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.534972 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.537010 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.543379 (+ 6369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.544319 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.544444 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.548372 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.548389 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.548405 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.548451 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.552417 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.553255 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.553264 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.553268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.553414 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.553824 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.553828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.553831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.554262 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.554549 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.555737 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.555953 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:54:08.557073 27370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:08.557534 27168 catalog_manager.cc:4841] TS 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): tablet 7c45a5029048454fa28952a02927079c (table test-table [id=72fa6c81fe704e1cb4a66e04d7b2ad46]) successfully deleted
I20260807 22:54:08.564045 27701 ts_tablet_manager.cc:1905] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:08.574354 27701 ts_tablet_manager.cc:1918] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:08.574566 27701 log.cc:1196] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/7c45a5029048454fa28952a02927079c
I20260807 22:54:08.575139 27701 ts_tablet_manager.cc:1939] T 7c45a5029048454fa28952a02927079c P e266ab6347834bcf8379e819178980c1: Deleting consensus metadata
I20260807 22:54:08.577167 27165 catalog_manager.cc:4841] TS e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): tablet 7c45a5029048454fa28952a02927079c (table test-table [id=72fa6c81fe704e1cb4a66e04d7b2ad46]) successfully deleted
I20260807 22:54:08.684715 27702 tablet_replica.cc:330] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: stopping tablet replica
I20260807 22:54:08.685209 27702 raft_consensus.cc:2229] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:08.685753 27702 raft_consensus.cc:2258] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:08.686764 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27218
W20260807 22:54:08.708549 27165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:39815 (error 108)
I20260807 22:54:08.709347 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27380
I20260807 22:54:08.710955 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.710377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38710)
0807 22:54:08.710668 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.710766 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260807 22:54:08.711241 27165 catalog_manager.cc:4584] TS 1cb23ff56e0646ef83852f3f7a62ca92 (127.9.194.193:39815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c45a5029048454fa28952a02927079c: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
I20260807 22:54:08.728308 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.710414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.9.194.194:41409)
0807 22:54:08.710616 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.710622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.710642 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:54:08.710708 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.710713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.710790 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.710906 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.712432 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.713682 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.714990 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.715860 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.716010 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.718595 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.718612 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.718633 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.718677 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.722641 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.723594 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.723605 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.723609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.723798 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.726243 (+ 2445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.726249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.726253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.726743 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.727077 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.727474 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.727723 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:54:08.729054 27509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:08.765115 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.764552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38722)
0807 22:54:08.764782 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.764923 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 22:54:08.766238 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.765815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38736)
0807 22:54:08.765986 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.766090 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:54:08.826771 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.826186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38744)
0807 22:54:08.826471 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.826589 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 22:54:08.827940 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.827521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38748)
0807 22:54:08.827672 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.827770 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 22:54:08.841881 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27380
I20260807 22:54:08.863183 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27518
I20260807 22:54:08.878664 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.863933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.9.194.195:39107)
0807 22:54:08.864109 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:08.864114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:08.864131 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:54:08.864286 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:08.864290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:08.864352 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:08.864437 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:08.865937 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.866683 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.868003 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:08.868392 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:08.868485 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:08.871176 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:08.871196 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:08.871220 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:08.871276 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:08.875290 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.876184 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.876198 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.876203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.876423 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:08.876884 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:08.876889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:08.876906 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:08.877395 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:08.877698 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:08.878077 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 22:54:08.878313 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:54:08.879369 27651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:08.909279 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.908715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38756)
0807 22:54:08.908964 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.909099 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:54:08.910440 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:08.909953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38758)
0807 22:54:08.910187 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:08.910280 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 22:54:08.968742 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27518
I20260807 22:54:08.988691 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27146
I20260807 22:54:08.989858 27211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:09.105413 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:09.104631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39815 (local address 127.0.0.1:38762)
0807 22:54:09.105034 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:09.105174 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260807 22:54:09.105439 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27146
I20260807 22:54:29.130577 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--webserver_interface=127.9.194.254
--webserver_port=45985
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:29.490607 27708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:29.490967 27708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:29.491160 27708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:29.498574 27708 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:54:29.498677 27708 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:54:29.498751 27708 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:54:29.498826 27708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:29.498929 27708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:29.499048 27708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:29.499130 27708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:29.499207 27708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:29.509760 27708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=45985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:29.511250 27708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:29.512702 27708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:29.525614 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:29.525697 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:29.525656 27717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:29.526429 27708 server_base.cc:1034] running on GCE node
I20260807 22:54:29.527244 27708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:29.528853 27708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:29.530144 27708 hybrid_clock.cc:648] HybridClock initialized: now 1786143269530053 us; error 56 us; skew 500 ppm
I20260807 22:54:29.530516 27708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:29 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143269, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:29.533908 27708 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:54:29.535542 27708 webserver.cc:460] Webserver started at http://127.9.194.254:45985/ using document root <none> and password file <none>
I20260807 22:54:29.536257 27708 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:29.536397 27708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:29.542008 27708 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:54:29.545003 27724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:29.546303 27708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:54:29.546496 27708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "8a9e67bee9fe488ba2c5b74915883421"
format_stamp: "Formatted at 2026-08-07 22:53:43 on dist-test-slave-5k54"
server_key: "2e78e037bb636923cd4e2a53a2f4a942"
server_key_iv: "44bc0df3f2dd74c3f0b60cce97df1881"
server_key_version: "encryptionkey@0"
I20260807 22:54:29.547533 27708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:29.562119 27708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:29.566177 27708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:29.566416 27708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:29.585999 27708 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:45469
I20260807 22:54:29.586018 27780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:45469 every 8 connection(s)
I20260807 22:54:29.587597 27708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:29.592412 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27708
I20260807 22:54:29.593405 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:39815
--local_ip_for_outbound_sockets=127.9.194.193
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=37145
--webserver_interface=127.9.194.193
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:54:29.595268 27781 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:54:29.598737 27781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap starting.
I20260807 22:54:29.620157 27781 log.cc:824] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:29.636759 27781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:54:29.637356 27781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Bootstrap complete.
I20260807 22:54:29.646706 27781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:29.647728 27781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Initialized, Role: FOLLOWER
I20260807 22:54:29.648401 27781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:29.648651 27781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:29.648785 27781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:29.648988 27781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:54:29.653294 27781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:29.653810 27781 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e67bee9fe488ba2c5b74915883421; no voters:
I20260807 22:54:29.654511 27781 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:54:29.654696 27786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:54:29.655920 27786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [term 3 LEADER]: Becoming Leader. State: Replica: 8a9e67bee9fe488ba2c5b74915883421, State: Running, Role: LEADER
I20260807 22:54:29.656500 27786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } }
I20260807 22:54:29.656837 27781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:29.659274 27787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:54:29.659759 27787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:29.663622 27788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e67bee9fe488ba2c5b74915883421. Latest consensus state: current_term: 3 leader_uuid: "8a9e67bee9fe488ba2c5b74915883421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e67bee9fe488ba2c5b74915883421" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 45469 } } }
I20260807 22:54:29.663986 27788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:29.665802 27792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:29.676357 27792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:29.678398 27792 catalog_manager.cc:1259] Loaded cluster ID: f0b91b432696475181e3a406c408ada9
I20260807 22:54:29.678537 27792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:29.683873 27792 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:29.686048 27792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Loaded TSK: 0
I20260807 22:54:29.686225 27792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a9e67bee9fe488ba2c5b74915883421: Loaded TSK: 1
F20260807 22:54:29.687690 27792 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa5a682f9ad google::LogMessage::Fail() at ??:0
@ 0x7fa5a683288a google::LogMessage::SendToLog() at ??:0
@ 0x7fa5a682f4de google::LogMessage::Flush() at ??:0
@ 0x7fa5a6830a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa5aa5937d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa5aa58273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa5bc25e1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa5bc25d783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa5bc2e92a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fa5bc2e90c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fa5bc3335e9 std::function<>::operator()() at ??:0
@ 0x7fa5bc261cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa5bc25fc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa5bc2e7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa5bc2e7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa5a947b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa5a947e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa5a947e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa5a9459eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa5ac7336db start_thread at ??:0
@ 0x7fa5a56c961f clone at ??:0
W20260807 22:54:29.992861 27783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:29.993198 27783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:29.993386 27783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:30.000736 27783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:30.000916 27783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:30.001099 27783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:30.011410 27783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:39815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=37145
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:30.012907 27783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:30.014307 27783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:30.028298 27831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.028373 27834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.028321 27832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:30.029196 27783 server_base.cc:1034] running on GCE node
I20260807 22:54:30.029960 27783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:30.031119 27783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:30.032367 27783 hybrid_clock.cc:648] HybridClock initialized: now 1786143270032307 us; error 54 us; skew 500 ppm
I20260807 22:54:30.032720 27783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:30 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:30.035974 27783 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:54:30.037523 27783 webserver.cc:460] Webserver started at http://127.9.194.193:37145/ using document root <none> and password file <none>
I20260807 22:54:30.038224 27783 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:30.038393 27783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:30.044028 27783 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:54:30.047014 27843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:30.048501 27783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:30.048717 27783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "1cb23ff56e0646ef83852f3f7a62ca92"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "bf9364736a479a9cecd85d694143361e"
server_key_iv: "2bd5e32148c48f5cdf5381371377058a"
server_key_version: "encryptionkey@0"
I20260807 22:54:30.049767 27783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:30.066099 27783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:30.070096 27783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:30.070379 27783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:30.071367 27783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:30.073199 27852 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 22:54:30.078733 27783 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:54:30.078889 27783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260807 22:54:30.079039 27783 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:54:30.082819 27783 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:54:30.082933 27783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.001s
I20260807 22:54:30.083406 27852 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Bootstrap starting.
I20260807 22:54:30.117619 27852 log.cc:824] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:30.122686 27783 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39815
I20260807 22:54:30.122896 27966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39815 every 8 connection(s)
I20260807 22:54:30.124169 27783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:54:30.132977 27852 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:54:30.133078 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.127219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:39111)
0807 22:54:30.130934 (+ 3715us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.131542 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3219,"thread_start_us":2930,"threads_started":1}
I20260807 22:54:30.133922 27852 tablet_bootstrap.cc:492] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Bootstrap complete.
W20260807 22:54:30.133949 27967 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:45469 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:45469: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
I20260807 22:54:30.134491 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27783
I20260807 22:54:30.135217 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:41409
--local_ip_for_outbound_sockets=127.9.194.194
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=36453
--webserver_interface=127.9.194.194
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:54:30.135305 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.134879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:59459)
0807 22:54:30.135057 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.135160 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:54:30.135888 27852 ts_tablet_manager.cc:1397] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent bootstrapping tablet: real 0.053s user 0.024s sys 0.026s
I20260807 22:54:30.136475 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.135932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:49721)
0807 22:54:30.136176 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.136291 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260807 22:54:30.136698 27967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:54:30.142519 27852 raft_consensus.cc:348] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:30.143473 27852 raft_consensus.cc:729] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb23ff56e0646ef83852f3f7a62ca92, State: Initialized, Role: FOLLOWER
I20260807 22:54:30.144080 27852 consensus_queue.cc:260] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:30.145670 27852 ts_tablet_manager.cc:1428] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260807 22:54:30.146023 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.145570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:33521)
0807 22:54:30.145772 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.145877 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 22:54:30.147032 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.146660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:56755)
0807 22:54:30.146795 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.146916 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:54:30.475816 27971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:30.476095 27971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:30.476276 27971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:30.483565 27971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:30.483678 27971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:30.483891 27971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:54:30.493834 27971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:41409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=36453
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:30.495333 27971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:30.497072 27971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:30.510797 27977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.510982 27979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.510975 27976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:30.512156 27971 server_base.cc:1034] running on GCE node
I20260807 22:54:30.512836 27971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:30.513947 27971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:30.515182 27971 hybrid_clock.cc:648] HybridClock initialized: now 1786143270515140 us; error 29 us; skew 500 ppm
I20260807 22:54:30.515586 27971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:30 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:30.518962 27971 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:54:30.520479 27971 webserver.cc:460] Webserver started at http://127.9.194.194:36453/ using document root <none> and password file <none>
I20260807 22:54:30.521136 27971 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:30.521238 27971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:30.526664 27971 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 22:54:30.529628 27986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:30.531248 27971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:30.531422 27971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "e266ab6347834bcf8379e819178980c1"
format_stamp: "Formatted at 2026-08-07 22:53:44 on dist-test-slave-5k54"
server_key: "ef7a69c02918e09696525b99ae7997fc"
server_key_iv: "80f7d9136ca2b9016e2cdee79ae29bae"
server_key_version: "encryptionkey@0"
I20260807 22:54:30.532593 27971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:30.565521 27971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:30.570242 27971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:30.570549 27971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:30.571705 27971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:30.573077 27971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:30.573199 27971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:30.573339 27971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:30.573442 27971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:30.607915 27971 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:41409
I20260807 22:54:30.608016 28102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:41409 every 8 connection(s)
I20260807 22:54:30.609412 27971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:54:30.614332 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.612518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:45241)
0807 22:54:30.613258 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.613492 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":282,"threads_started":1}
W20260807 22:54:30.615072 28103 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:45469 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:45469: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
I20260807 22:54:30.615648 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 27971
I20260807 22:54:30.616113 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.615687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:60577)
0807 22:54:30.615891 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.615973 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:54:30.616524 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:39107
--local_ip_for_outbound_sockets=127.9.194.195
--tserver_master_addrs=127.9.194.254:45469
--webserver_port=40809
--webserver_interface=127.9.194.195
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:54:30.617234 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:30.616808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:60411)
0807 22:54:30.616975 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:30.617083 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 22:54:30.617498 28103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:54:30.952606 28106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:30.952936 28106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:30.953120 28106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:30.960479 28106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:30.960583 28106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:30.960691 28106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:54:30.970350 28106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:39107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=40809
--tserver_master_addrs=127.9.194.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:30.971601 28106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:30.972904 28106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:30.986418 28111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.986338 28113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:30.986459 28115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:30.987254 28106 server_base.cc:1034] running on GCE node
I20260807 22:54:30.987892 28106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:30.988852 28106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:30.990051 28106 hybrid_clock.cc:648] HybridClock initialized: now 1786143270989991 us; error 54 us; skew 500 ppm
I20260807 22:54:30.990389 28106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:30 dist-test-slave-5k54 krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:30.993554 28106 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:54:30.995168 28106 webserver.cc:460] Webserver started at http://127.9.194.195:40809/ using document root <none> and password file <none>
I20260807 22:54:30.995911 28106 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:30.996057 28106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:31.001299 28106 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:54:31.004103 28123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:31.005515 28106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:31.005700 28106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "020629862db64f65bf5c1d4323c4ab47"
format_stamp: "Formatted at 2026-08-07 22:53:45 on dist-test-slave-5k54"
server_key: "cf7414f486ecb1d3c767f42fdc3f8cb5"
server_key_iv: "9c01b9f8b59312c0d7f0e2abd42136c1"
server_key_version: "encryptionkey@0"
I20260807 22:54:31.006699 28106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:31.025405 28106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:31.029795 28106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:31.030086 28106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:31.031075 28106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:31.032560 28106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:31.032647 28106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:31.032802 28106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:31.032905 28106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:31.066509 28106 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39107
I20260807 22:54:31.066594 28243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39107 every 8 connection(s)
I20260807 22:54:31.068079 28106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:54:31.072881 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.071047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:43971)
0807 22:54:31.071809 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.072030 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":360,"threads_started":1}
W20260807 22:54:31.073622 28244 heartbeater.cc:644] Failed to heartbeat to 127.9.194.254:45469 (0 consecutive failures): Network error: Failed to ping master at 127.9.194.254:45469: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
I20260807 22:54:31.074390 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.074076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:43843)
0807 22:54:31.074225 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.074296 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 22:54:31.075223 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.074899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:46073)
0807 22:54:31.075043 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.075123 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:54:31.075517 28244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:54:31.076454 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28106
I20260807 22:54:31.148677 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.147882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:56191)
0807 22:54:31.148373 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.148465 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":240,"threads_started":1}
I20260807 22:54:31.618848 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.618096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:49717)
0807 22:54:31.618542 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.618629 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":223,"threads_started":1}
I20260807 22:54:31.720358 28251 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:31.720676 28251 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:31.722432 28251 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
Aug 07 22:54:31 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
Aug 07 22:54:31 dist-test-slave-5k54 krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:54:31.759601 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.722690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39107 (local address 127.9.194.193:37529)
0807 22:54:31.735757 (+ 13067us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.735774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:31.735896 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:31.739484 (+ 3588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:31.739496 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:31.740714 (+ 1218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:31.742070 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:31.742099 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.744571 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.744581 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:31.746118 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:31.746132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.747170 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.747176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:31.747512 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:31.748299 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:31.748327 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:31.751084 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:31.754930 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:31.754940 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:31.754956 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:31.755701 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:31.756287 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:31.756292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:31.756296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:31.756593 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:31.757597 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:31.757605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:31.758140 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 22:54:31.758471 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 22:54:31.759140 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12958,"mutex_wait_us":503,"spinlock_wait_cycles":46208,"thread_start_us":11858,"threads_started":1}
I20260807 22:54:31.759677 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.723012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:37529 (local address 127.9.194.195:39107)
0807 22:54:31.735375 (+ 12363us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:31.735394 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:31.735972 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:54:31.738329 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:31.738343 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:31.738890 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:31.739269 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:31.742342 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.744290 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.746273 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.746939 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.747144 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:31.751355 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:31.751362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:31.751374 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:31.751472 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:31.754767 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:31.755870 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:31.755877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:31.755909 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:31.756138 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:31.756732 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:31.756737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:31.756741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:31.757444 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:31.757636 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:31.758771 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0807 22:54:31.759212 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12101,"thread_start_us":11949,"threads_started":1}
I20260807 22:54:31.759689 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.723012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:45927 (local address 127.9.194.194:41409)
0807 22:54:31.734466 (+ 11454us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:31.734492 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:31.735249 (+ 757us) server_negotiation.cc:407] Connection header received
0807 22:54:31.738358 (+ 3109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:31.738367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:31.738890 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:31.739269 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:31.742342 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.744290 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.746275 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.746927 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.747147 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:31.751278 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:31.751287 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:31.751302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:31.751348 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:31.755099 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:31.755985 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:31.755993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:31.756002 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:31.756187 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:31.756732 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:31.756737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:31.756741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:31.757445 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:31.757638 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:31.758773 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0807 22:54:31.759215 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11188,"thread_start_us":10991,"threads_started":1}
I20260807 22:54:31.759641 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:31.722661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41409 (local address 127.9.194.193:45927)
0807 22:54:31.734822 (+ 12161us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:31.734887 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:31.735667 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:31.739484 (+ 3817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:31.739496 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:31.740714 (+ 1218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:31.742070 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:31.742099 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.744518 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.744525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:31.746116 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:31.746129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:31.747156 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:31.747162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:31.747506 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:31.748361 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:31.748378 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:31.750957 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:31.755253 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:31.755259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:31.755263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:31.755825 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:31.756289 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:31.756293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:31.756296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:31.756593 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:31.757597 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:31.757605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:31.758140 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 22:54:31.758471 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 22:54:31.759032 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11958,"mutex_wait_us":348,"spinlock_wait_cycles":28032,"thread_start_us":12786,"threads_started":1}
W20260807 22:54:31.762916 27849 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
W20260807 22:54:31.763221 27844 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
I20260807 22:54:31.763393 27844 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92; no voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1
I20260807 22:54:31.763875 28251 raft_consensus.cc:2726] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:54:32.076973 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:32.076115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:44833)
0807 22:54:32.076587 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:32.076729 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":216,"threads_started":1}
I20260807 22:54:32.150089 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:32.149631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:54451)
0807 22:54:32.149830 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:32.149935 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 22:54:32.620563 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:32.619854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:52993)
0807 22:54:32.620294 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:32.620374 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260807 22:54:33.078543 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:33.077900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:49609)
0807 22:54:33.078304 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:33.078383 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":222,"threads_started":1}
I20260807 22:54:33.151961 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:33.151033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:59049)
0807 22:54:33.151641 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:33.151736 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":239,"threads_started":1}
I20260807 22:54:33.622067 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:33.621367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:58677)
0807 22:54:33.621819 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:33.621896 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":248,"threads_started":1}
I20260807 22:54:33.654402 28263 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:33.654651 28263 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:33.655336 28263 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
W20260807 22:54:33.656476 27849 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
W20260807 22:54:33.656670 27844 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
I20260807 22:54:33.656836 27844 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92; no voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1
I20260807 22:54:33.657107 28263 raft_consensus.cc:2726] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:54:34.080185 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:34.079479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:37691)
0807 22:54:34.079917 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:34.080026 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":220,"threads_started":1}
I20260807 22:54:34.153919 28265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:34.152876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:36473)
0807 22:54:34.153361 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:34.153629 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":241,"threads_started":1}
I20260807 22:54:34.624009 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:34.623039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:41039)
0807 22:54:34.623705 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:34.623810 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":269,"threads_started":1}
I20260807 22:54:35.081928 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:35.081117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:35237)
0807 22:54:35.081661 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:35.081745 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":317,"threads_started":1}
I20260807 22:54:35.155704 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:35.154968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:33059)
0807 22:54:35.155385 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:35.155507 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":225,"threads_started":1}
I20260807 22:54:35.625746 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:35.625019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:57111)
0807 22:54:35.625501 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:35.625581 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":239,"threads_started":1}
I20260807 22:54:35.867867 28270 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:35.868060 28270 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:35.868649 28270 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
W20260807 22:54:35.869742 27849 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
W20260807 22:54:35.869944 27844 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
I20260807 22:54:35.870086 27844 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92; no voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1
I20260807 22:54:35.870391 28270 raft_consensus.cc:2726] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:54:36.083647 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:36.082841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:47387)
0807 22:54:36.083331 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:36.083417 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":242,"threads_started":1}
I20260807 22:54:36.157409 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:36.156599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:44413)
0807 22:54:36.157127 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:36.157220 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":234,"threads_started":1}
I20260807 22:54:36.627897 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:36.626884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:44789)
0807 22:54:36.627519 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:36.627657 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":318,"threads_started":1}
I20260807 22:54:37.085217 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:37.084505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:48829)
0807 22:54:37.084969 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:37.085049 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260807 22:54:37.159056 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:37.158272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:50619)
0807 22:54:37.158702 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:37.158792 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":236,"threads_started":1}
I20260807 22:54:37.629596 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:37.628801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:44469)
0807 22:54:37.629314 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:37.629409 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":236,"threads_started":1}
I20260807 22:54:38.087082 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:38.086286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:45233)
0807 22:54:38.086806 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:38.086901 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":242,"threads_started":1}
I20260807 22:54:38.160862 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:38.160052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:44761)
0807 22:54:38.160602 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:38.160694 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":283,"threads_started":1}
I20260807 22:54:38.504174 28279 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:38.504446 28279 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:38.505174 28279 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
W20260807 22:54:38.506498 27844 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
W20260807 22:54:38.506702 27849 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
I20260807 22:54:38.506821 27849 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92; no voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1
I20260807 22:54:38.507048 28279 raft_consensus.cc:2726] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:54:38.631507 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:38.630737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:60331)
0807 22:54:38.631187 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:38.631287 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260807 22:54:39.088696 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:39.087967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:43429)
0807 22:54:39.088418 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:39.088503 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":245,"threads_started":1}
I20260807 22:54:39.162638 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:39.161890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:55579)
0807 22:54:39.162369 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:39.162456 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":257,"threads_started":1}
I20260807 22:54:39.633061 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:39.632355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:38425)
0807 22:54:39.632805 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:39.632890 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":221,"threads_started":1}
I20260807 22:54:40.090476 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:40.089736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:49199)
0807 22:54:40.090199 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:40.090293 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260807 22:54:40.164386 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:40.163676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:44449)
0807 22:54:40.164119 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:40.164205 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260807 22:54:40.634651 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:40.633979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:50193)
0807 22:54:40.634400 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:40.634481 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":230,"threads_started":1}
I20260807 22:54:41.092523 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:41.091655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:39289)
0807 22:54:41.092225 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:41.092326 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":358,"threads_started":1}
I20260807 22:54:41.166033 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:41.165275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:43977)
0807 22:54:41.165716 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:41.165808 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260807 22:54:41.636731 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:41.635806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:59243)
0807 22:54:41.636468 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:41.636564 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":391,"threads_started":1}
I20260807 22:54:42.094273 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:42.093530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:37389)
0807 22:54:42.094033 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:42.094117 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":222,"threads_started":1}
I20260807 22:54:42.167770 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:42.166971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:41227)
0807 22:54:42.167511 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:42.167600 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260807 22:54:42.638499 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:42.637743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:38269)
0807 22:54:42.638246 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:42.638326 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":293,"threads_started":1}
I20260807 22:54:43.096112 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:43.095182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:52855)
0807 22:54:43.095781 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:43.095896 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":312,"threads_started":1}
I20260807 22:54:43.169505 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:43.168663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:40615)
0807 22:54:43.169163 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:43.169274 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":281,"threads_started":1}
I20260807 22:54:43.640139 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:43.639364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:55797)
0807 22:54:43.639869 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:43.639963 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":284,"threads_started":1}
I20260807 22:54:43.987974 28297 raft_consensus.cc:482] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:43.988200 28297 raft_consensus.cc:504] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e266ab6347834bcf8379e819178980c1" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41409 } } peers { permanent_uuid: "020629862db64f65bf5c1d4323c4ab47" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39107 } } peers { permanent_uuid: "1cb23ff56e0646ef83852f3f7a62ca92" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39815 } }
I20260807 22:54:43.989030 28297 leader_election.cc:288] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409), 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107)
W20260807 22:54:43.990005 27844 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e266ab6347834bcf8379e819178980c1 (127.9.194.194:41409): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
W20260807 22:54:43.990177 27849 leader_election.cc:341] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 020629862db64f65bf5c1d4323c4ab47 (127.9.194.195:39107): Not found: Tablet not found: 7c45a5029048454fa28952a02927079c
I20260807 22:54:43.990350 27849 leader_election.cc:302] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb23ff56e0646ef83852f3f7a62ca92; no voters: 020629862db64f65bf5c1d4323c4ab47, e266ab6347834bcf8379e819178980c1
I20260807 22:54:43.990597 28297 raft_consensus.cc:2726] T 7c45a5029048454fa28952a02927079c P 1cb23ff56e0646ef83852f3f7a62ca92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:54:44.097839 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:44.097110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:51607)
0807 22:54:44.097588 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:44.097679 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":252,"threads_started":1}
I20260807 22:54:44.171262 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:44.170378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:60727)
0807 22:54:44.170898 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:44.171010 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":268,"threads_started":1}
I20260807 22:54:44.641871 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:44.641136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:55801)
0807 22:54:44.641594 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:44.641688 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":251,"threads_started":1}
I20260807 22:54:45.099699 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:45.098852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:34541)
0807 22:54:45.099348 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:45.099499 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":288,"threads_started":1}
I20260807 22:54:45.173156 28302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:45.172439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.193:58633)
0807 22:54:45.172889 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:45.172981 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260807 22:54:45.643790 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:45.642962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.194:44261)
0807 22:54:45.643527 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:45.643610 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":247,"threads_started":1}
I20260807 22:54:46.080787 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27783
I20260807 22:54:46.101452 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:46.081853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.9.194.193:39815)
0807 22:54:46.082487 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:46.082497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:46.082695 (+ 198us) server_negotiation.cc:407] Connection header received
0807 22:54:46.082949 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:46.082960 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:46.083214 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:46.083388 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:46.084815 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.085760 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.087077 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.087756 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.087897 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:46.095675 (+ 7778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:46.095689 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:46.095703 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:46.095740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:46.098494 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.099223 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.099230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.099232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.099384 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.099800 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.099805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.099808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.100205 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:46.100526 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:46.100820 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 22:54:46.101103 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":288,"threads_started":1}
I20260807 22:54:46.101904 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:46.100802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:45469 (local address 127.9.194.195:60297)
0807 22:54:46.101495 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:46.101663 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.254:45469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":304,"threads_started":1}
I20260807 22:54:46.102861 27962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:46.195641 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27783
I20260807 22:54:46.213085 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 27971
I20260807 22:54:46.226670 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:46.213791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.9.194.194:41409)
0807 22:54:46.214236 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:46.214241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:46.214256 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:54:46.214310 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:46.214314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:46.214371 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:46.214464 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:46.215993 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.216781 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.218028 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.218452 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.218543 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:46.221299 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:46.221304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:46.221307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:46.221341 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:46.224149 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.224840 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.224844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.224846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.225004 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.225382 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.225385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.225387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.225643 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:46.225842 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:46.226108 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:54:46.226266 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":247,"threads_started":1}
I20260807 22:54:46.227497 28098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:46.312496 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 27971
I20260807 22:54:46.333118 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28106
I20260807 22:54:46.346676 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:46.333991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.9.194.195:39107)
0807 22:54:46.334577 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:46.334581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:46.334598 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:54:46.334653 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:46.334656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:46.334725 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:46.334817 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:46.336216 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.337008 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.338136 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:46.338544 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:46.338647 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:46.341275 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:46.341280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:46.341302 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:46.341336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:46.344085 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.344780 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.344799 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.344809 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.344970 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:46.345335 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:46.345339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:46.345342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:46.345613 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:46.345832 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:46.346081 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 22:54:46.346241 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":306,"threads_started":1}
I20260807 22:54:46.347384 28238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:46.431885 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28106
2026-08-07T22:54:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64486 ms)
[----------] 30 tests from SecurityITest (302657 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:54:46 dist-test-slave-5k54 krb5kdc[28315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:54:46 dist-test-slave-5k54 krb5kdc[28315](info): set up 2 sockets
Aug 07 22:54:46 dist-test-slave-5k54 krb5kdc[28315](info): commencing operation
krb5kdc: starting...
W20260807 22:54:47.600868 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:54:47 dist-test-slave-5k54 krb5kdc[28315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:54:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:54:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:54:47.914489 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36487
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:48.256192 28329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:48.256480 28329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:48.256652 28329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:48.263818 28329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:48.263960 28329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:48.264065 28329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:48.264144 28329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:48.264207 28329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:48.274104 28329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:48.275466 28329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:48.276793 28329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:48.289032 28337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:48.289057 28335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:48.289165 28334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:48.290257 28329 server_base.cc:1034] running on GCE node
I20260807 22:54:48.291011 28329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:48.292419 28329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:48.293627 28329 hybrid_clock.cc:648] HybridClock initialized: now 1786143288293564 us; error 49 us; skew 500 ppm
I20260807 22:54:48.293928 28329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:48 dist-test-slave-5k54 krb5kdc[28315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:48.297011 28329 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:54:48.298436 28329 webserver.cc:460] Webserver started at http://127.9.194.254:34433/ using document root <none> and password file <none>
I20260807 22:54:48.299113 28329 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:48.299256 28329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:48.299618 28329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:48.302629 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "75c42853da794a8d9a8c5e7445ec6c22"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "8ab81109c6291f3d5b69c6ce8ad9e2a7"
server_key_iv: "32428f7ef09e44f7e92fdb9b844da4ee"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.303388 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "75c42853da794a8d9a8c5e7445ec6c22"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "8ab81109c6291f3d5b69c6ce8ad9e2a7"
server_key_iv: "32428f7ef09e44f7e92fdb9b844da4ee"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.308615 28329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:54:48.312161 28345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:48.313488 28329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:54:48.313686 28329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "75c42853da794a8d9a8c5e7445ec6c22"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "8ab81109c6291f3d5b69c6ce8ad9e2a7"
server_key_iv: "32428f7ef09e44f7e92fdb9b844da4ee"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.313895 28329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:48.332758 28329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:48.336962 28329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:48.337239 28329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:48.356626 28329 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:36487
I20260807 22:54:48.356649 28401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:36487 every 8 connection(s)
I20260807 22:54:48.358050 28329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:48.361768 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28329
I20260807 22:54:48.362010 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:54:48.362110 28402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:48.362723 9995 external_mini_cluster.cc:1428] Setting key a0923b23ec0335177143ece4a0f3c88d
I20260807 22:54:48.376078 28402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: Bootstrap starting.
Aug 07 22:54:48 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:48.380029 28402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:48.381428 28402 log.cc:824] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:48.384832 28402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: No bootstrap required, opened a new log
I20260807 22:54:48.388701 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:48.364738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.9.194.254:36487)
0807 22:54:48.365590 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:48.365601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:48.365640 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:54:48.368184 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:48.368208 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:48.368623 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:48.368966 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:48.370792 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.372364 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.374234 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.374916 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.375045 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:48.380052 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:48.380064 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:48.380084 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:48.380133 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:48.384126 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:48.384858 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:48.384867 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:48.384883 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:48.385097 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:48.385503 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:48.385508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:48.385513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:48.386199 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:48.386475 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:48.386883 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 22:54:48.387333 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":400,"threads_started":1}
I20260807 22:54:48.394078 28402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } }
I20260807 22:54:48.394402 28402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:48.394474 28402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c42853da794a8d9a8c5e7445ec6c22, State: Initialized, Role: FOLLOWER
I20260807 22:54:48.394981 28402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } }
I20260807 22:54:48.395184 28402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:48.395282 28402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:48.395408 28402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:48.397629 28402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } }
I20260807 22:54:48.398167 28402 leader_election.cc:302] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c42853da794a8d9a8c5e7445ec6c22; no voters:
I20260807 22:54:48.398722 28402 leader_election.cc:288] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:54:48.398895 28408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:48.399616 28408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [term 1 LEADER]: Becoming Leader. State: Replica: 75c42853da794a8d9a8c5e7445ec6c22, State: Running, Role: LEADER
I20260807 22:54:48.400213 28408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } }
I20260807 22:54:48.400794 28402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:48.403141 28410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c42853da794a8d9a8c5e7445ec6c22. Latest consensus state: current_term: 1 leader_uuid: "75c42853da794a8d9a8c5e7445ec6c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } } }
I20260807 22:54:48.403618 28410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:48.404284 28409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75c42853da794a8d9a8c5e7445ec6c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c42853da794a8d9a8c5e7445ec6c22" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 36487 } } }
I20260807 22:54:48.404644 28409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:48.407579 28416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:48.411566 28416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:48.420507 28416 catalog_manager.cc:1347] Generated new cluster ID: 1207057c294049c38be3727d4775b702
I20260807 22:54:48.420629 28416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:48.442229 28416 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:54:48.443638 28416 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:48.449818 28416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: Generated new TSK 0
I20260807 22:54:48.450629 28416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:54:48.524348 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:48.860486 28430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:48.860772 28430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:48.860891 28430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:48.868036 28430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:48.868141 28430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:48.868254 28430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:48.879292 28430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:48.880632 28430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:48.881894 28430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:48.895097 28435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:48.895144 28438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:48.895184 28436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:48.895995 28430 server_base.cc:1034] running on GCE node
I20260807 22:54:48.896518 28430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:48.897365 28430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:48.898555 28430 hybrid_clock.cc:648] HybridClock initialized: now 1786143288898514 us; error 30 us; skew 500 ppm
I20260807 22:54:48.898818 28430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:48 dist-test-slave-5k54 krb5kdc[28315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:48.902060 28430 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:54:48.903626 28430 webserver.cc:460] Webserver started at http://127.9.194.193:35195/ using document root <none> and password file <none>
I20260807 22:54:48.904287 28430 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:48.904431 28430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:48.904736 28430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:48.907944 28430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "59f6e21e7e28429ab4cb4f53101fb851"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "738efa56a5d425bd7d2bf36c5336c9e1"
server_key_iv: "d1b4a9e7466ceb6840b7eea8a8bce061"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.908775 28430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "59f6e21e7e28429ab4cb4f53101fb851"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "738efa56a5d425bd7d2bf36c5336c9e1"
server_key_iv: "d1b4a9e7466ceb6840b7eea8a8bce061"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.913812 28430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:54:48.917295 28445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:48.918665 28430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:48.918838 28430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "59f6e21e7e28429ab4cb4f53101fb851"
format_stamp: "Formatted at 2026-08-07 22:54:48 on dist-test-slave-5k54"
server_key: "738efa56a5d425bd7d2bf36c5336c9e1"
server_key_iv: "d1b4a9e7466ceb6840b7eea8a8bce061"
server_key_version: "encryptionkey@0"
I20260807 22:54:48.919036 28430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:48.941468 28430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:48.945498 28430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:48.945788 28430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:48.946764 28430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:48.948091 28430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:48.948169 28430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:48.948324 28430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:48.948396 28430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:48.981309 28430 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:39575
I20260807 22:54:48.981417 28566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:39575 every 8 connection(s)
I20260807 22:54:48.982604 28430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:54:48.992579 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28430
I20260807 22:54:48.992789 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:54:48.993443 9995 external_mini_cluster.cc:1428] Setting key 59a4d07c8ffe0f975701d946791ce3cb
Aug 07 22:54:48 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:49.003738 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:48.985201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:39353 (local address 127.9.194.254:36487)
0807 22:54:48.985643 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:48.985648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:48.986641 (+ 993us) server_negotiation.cc:407] Connection header received
0807 22:54:48.988601 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:48.988606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:48.988700 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:48.988815 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:48.990274 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.991002 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.992589 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.993087 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.993227 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:48.997647 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:48.997652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:48.997655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:48.997686 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:49.000246 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:49.001182 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:49.001187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:49.001189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:49.001344 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:49.001934 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:49.001938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:49.001940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:49.002222 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:49.002339 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:49.003230 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 22:54:49.003407 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":227,"threads_started":1}
I20260807 22:54:49.005205 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:48.985779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36487 (local address 127.9.194.193:39353)
0807 22:54:48.986479 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:48.986521 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:48.988339 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:48.989051 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:48.989060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:48.989558 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:48.990104 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:48.990125 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.991168 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.991172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:48.992437 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:48.992449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:48.993313 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:48.993321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:48.993602 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:48.994302 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:48.994332 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:48.997410 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:49.000386 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:49.000395 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:49.000411 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:49.001061 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:49.001489 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:49.001496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:49.001503 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:49.001747 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:49.002347 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:49.002357 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:49.002805 (+ 448us) client_negotiation.cc:769] Sending connection context
0807 22:54:49.003122 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 22:54:49.003724 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":269,"threads_started":1}
I20260807 22:54:49.006749 28567 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36487
I20260807 22:54:49.007067 28567 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:49.007807 28567 heartbeater.cc:511] Master 127.9.194.254:36487 requested a full tablet report, sending...
I20260807 22:54:49.009929 28364 ts_manager.cc:194] Registered new tserver with Master: 59f6e21e7e28429ab4cb4f53101fb851 (127.9.194.193:39575)
I20260807 22:54:49.011332 28364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:39353
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:54:49.067289 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:49.404853 28573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:49.405133 28573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:49.405226 28573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:49.412874 28573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:49.412986 28573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:49.413097 28573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:54:49.422921 28573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:49.424309 28573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:49.425580 28573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:49.438498 28578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:49.438498 28579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:49.438705 28582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:49.439930 28573 server_base.cc:1034] running on GCE node
I20260807 22:54:49.440553 28573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:49.441483 28573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:49.442708 28573 hybrid_clock.cc:648] HybridClock initialized: now 1786143289442655 us; error 46 us; skew 500 ppm
I20260807 22:54:49.443019 28573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:49 dist-test-slave-5k54 krb5kdc[28315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:49.446218 28573 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:54:49.447760 28573 webserver.cc:460] Webserver started at http://127.9.194.194:45327/ using document root <none> and password file <none>
I20260807 22:54:49.448432 28573 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:49.448576 28573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:49.448884 28573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:49.451872 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "88ce4622b78041d7abfe5f66017bfaec"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "a79196b1aa5bb67399b3372e2ca7e564"
server_key_iv: "62a9d3a501abf3f5cc6787df80ed9886"
server_key_version: "encryptionkey@0"
I20260807 22:54:49.452661 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "88ce4622b78041d7abfe5f66017bfaec"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "a79196b1aa5bb67399b3372e2ca7e564"
server_key_iv: "62a9d3a501abf3f5cc6787df80ed9886"
server_key_version: "encryptionkey@0"
I20260807 22:54:49.457726 28573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 22:54:49.461129 28589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:49.462411 28573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:54:49.462577 28573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "88ce4622b78041d7abfe5f66017bfaec"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "a79196b1aa5bb67399b3372e2ca7e564"
server_key_iv: "62a9d3a501abf3f5cc6787df80ed9886"
server_key_version: "encryptionkey@0"
I20260807 22:54:49.462769 28573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:49.492343 28573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:49.496405 28573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:49.496691 28573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:49.497648 28573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:49.499166 28573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:49.499291 28573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:49.499492 28573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:49.499571 28573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 22:54:49.532987 28573 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:33155
I20260807 22:54:49.533061 28707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:33155 every 8 connection(s)
I20260807 22:54:49.534263 28573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:54:49.540652 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28573
I20260807 22:54:49.540880 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:54:49.541298 9995 external_mini_cluster.cc:1428] Setting key 8dbbbc9b80719c59b3991d04068dcf4e
Aug 07 22:54:49 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:49.557479 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:49.537108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:35481 (local address 127.9.194.254:36487)
0807 22:54:49.537622 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:49.537629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:49.538611 (+ 982us) server_negotiation.cc:407] Connection header received
0807 22:54:49.540649 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:49.540653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:49.540723 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:49.540862 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:49.542818 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:49.543844 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:49.546677 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:49.547125 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:49.547222 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:49.551664 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:49.551669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:49.551672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:49.551704 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:49.554389 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:49.555105 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:49.555112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:49.555115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:49.555246 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:49.555733 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:49.555738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:49.555742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:49.556008 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:49.556159 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:49.556849 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 22:54:49.557075 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":237,"threads_started":1}
I20260807 22:54:49.558463 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:49.537726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36487 (local address 127.9.194.194:35481)
0807 22:54:49.538457 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:49.538493 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:49.540330 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:49.541249 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:49.541257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:49.541852 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:49.542648 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:49.542675 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:49.544020 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:49.544026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:49.546519 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:49.546538 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:49.547362 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:49.547372 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:49.547785 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:49.548552 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:49.548601 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:49.551385 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:49.554529 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:49.554540 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:49.554558 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:49.554990 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:49.555353 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:49.555358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:49.555362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:49.555615 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:49.556132 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:49.556140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:49.556485 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 22:54:49.556739 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 22:54:49.557284 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":288,"threads_started":1}
I20260807 22:54:49.560168 28708 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36487
I20260807 22:54:49.560483 28708 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:49.561196 28708 heartbeater.cc:511] Master 127.9.194.254:36487 requested a full tablet report, sending...
I20260807 22:54:49.562879 28364 ts_manager.cc:194] Registered new tserver with Master: 88ce4622b78041d7abfe5f66017bfaec (127.9.194.194:33155)
I20260807 22:54:49.564314 28364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:35481
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:54:49.612633 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:49.948127 28715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:49.948391 28715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:49.948483 28715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:49.955497 28715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:49.955595 28715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:49.955698 28715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:54:49.965034 28715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:49.966300 28715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:49.967716 28715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:49.980470 28723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:49.980520 28721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:49.980871 28720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:49.981673 28715 server_base.cc:1034] running on GCE node
I20260807 22:54:49.982288 28715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:49.983268 28715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:49.984484 28715 hybrid_clock.cc:648] HybridClock initialized: now 1786143289984436 us; error 43 us; skew 500 ppm
I20260807 22:54:49.984795 28715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:49 dist-test-slave-5k54 krb5kdc[28315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:49.987988 28715 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:54:49.989391 28715 webserver.cc:460] Webserver started at http://127.9.194.195:46751/ using document root <none> and password file <none>
I20260807 22:54:49.990042 28715 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:49.990166 28715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:49.990483 28715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:49.993299 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "2342b9f0de464dda93c497876d937b17"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "118028c6ff99186487885adea72f35ac"
server_key_iv: "d44739bbe53e75a38586d189422a06b8"
server_key_version: "encryptionkey@0"
I20260807 22:54:49.994089 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "2342b9f0de464dda93c497876d937b17"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "118028c6ff99186487885adea72f35ac"
server_key_iv: "d44739bbe53e75a38586d189422a06b8"
server_key_version: "encryptionkey@0"
I20260807 22:54:49.999135 28715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:54:50.002851 28730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:50.004462 28715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:50.004637 28715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "2342b9f0de464dda93c497876d937b17"
format_stamp: "Formatted at 2026-08-07 22:54:49 on dist-test-slave-5k54"
server_key: "118028c6ff99186487885adea72f35ac"
server_key_iv: "d44739bbe53e75a38586d189422a06b8"
server_key_version: "encryptionkey@0"
I20260807 22:54:50.004869 28715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:50.014263 28567 heartbeater.cc:503] Master 127.9.194.254:36487 was elected leader, sending a full tablet report...
I20260807 22:54:50.026515 28715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:50.030896 28715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:50.031206 28715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:50.032383 28715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:50.033869 28715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:50.033948 28715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:50.034124 28715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:50.034180 28715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:50.069060 28715 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:36195
I20260807 22:54:50.069149 28851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:36195 every 8 connection(s)
I20260807 22:54:50.070385 28715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:54:50.071873 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28715
I20260807 22:54:50.072086 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:54:50.072538 9995 external_mini_cluster.cc:1428] Setting key 3baa02ecd5b3324eada270f48d051f86
Aug 07 22:54:50 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:50.090778 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.073058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:57739 (local address 127.9.194.254:36487)
0807 22:54:50.073531 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.073536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.074701 (+ 1165us) server_negotiation.cc:407] Connection header received
0807 22:54:50.077034 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.077038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.077102 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.077190 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:50.078624 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.079383 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.081155 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.081549 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.081640 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.084939 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:50.084943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:50.084946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:50.084984 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:50.087777 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.088510 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.088516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.088520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.088642 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.089052 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.089059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.089063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.089336 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:50.089495 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.090147 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.090369 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260807 22:54:50.091790 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.073649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:36487 (local address 127.9.194.195:57739)
0807 22:54:50.074513 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.074589 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.076792 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.077408 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.077416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.077924 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:50.078487 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.078503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.079569 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.079573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.081031 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.081044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.081735 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.081740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.081928 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.082534 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:50.082561 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:50.084745 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:50.087940 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:50.087948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:50.087964 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:50.088378 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:50.088748 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:50.088753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:50.088756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:50.088939 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:50.089458 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:50.089466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:50.089796 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.090068 (+ 272us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.090583 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":368,"threads_started":1}
I20260807 22:54:50.093094 28852 heartbeater.cc:347] Connected to a master server at 127.9.194.254:36487
I20260807 22:54:50.093407 28852 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:50.094127 28852 heartbeater.cc:511] Master 127.9.194.254:36487 requested a full tablet report, sending...
I20260807 22:54:50.095578 28364 ts_manager.cc:194] Registered new tserver with Master: 2342b9f0de464dda93c497876d937b17 (127.9.194.195:36195)
I20260807 22:54:50.096359 28364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:57739
I20260807 22:54:50.100540 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:54:50.118201 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.105178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.9.194.254:36487)
0807 22:54:50.105329 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.105334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.105902 (+ 568us) server_negotiation.cc:407] Connection header received
0807 22:54:50.106054 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.106058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.106152 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.106245 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:50.107712 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.108519 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.110079 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.110508 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.110602 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.112990 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:50.112994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:50.112997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:50.113029 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:50.115683 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.116333 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.116337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.116339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.116458 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.116764 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.116767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.116769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.117142 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:50.117374 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.117746 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.117955 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:54:50.122262 28364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:54:50.129442 28364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:54:50.154654 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.141589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.9.194.194:33155)
0807 22:54:50.142950 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.142959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.144408 (+ 1449us) server_negotiation.cc:407] Connection header received
0807 22:54:50.145247 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.145256 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.145556 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.145764 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:50.146951 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.148708 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.151189 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.152848 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153008 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.153235 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.154055 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.154248 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1183,"thread_start_us":1072,"threads_started":1}
I20260807 22:54:50.154717 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.143783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:36195 (local address 127.0.0.1:45428)
0807 22:54:50.146533 (+ 2750us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.146559 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.146694 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.147479 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.147485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.147510 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:50.147830 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.147840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.149968 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.149975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.151919 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.151930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.153709 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.153905 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.153951 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.154041 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.154252 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2581,"mutex_wait_us":7,"thread_start_us":735,"threads_started":1}
I20260807 22:54:50.155613 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.141331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:33155 (local address 127.0.0.1:53238)
0807 22:54:50.144206 (+ 2875us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.144266 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.144956 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.145764 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.145773 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.145817 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:50.146747 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.146765 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.148896 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.148903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.151023 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.151036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.153384 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153394 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.153692 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.153753 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.154963 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.155116 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2561,"mutex_wait_us":426,"thread_start_us":4814,"threads_started":1}
I20260807 22:54:50.155808 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.144531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45428 (local address 127.9.194.195:36195)
0807 22:54:50.146840 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.146853 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.146902 (+ 49us) server_negotiation.cc:407] Connection header received
0807 22:54:50.146997 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.147007 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.147245 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.147488 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:50.148015 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.149788 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.152059 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.153438 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153559 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.153779 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.155126 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.155362 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2104,"thread_start_us":1916,"threads_started":1}
I20260807 22:54:50.157004 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.142209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.9.194.193:39575)
0807 22:54:50.143535 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.143547 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.145228 (+ 1681us) server_negotiation.cc:407] Connection header received
0807 22:54:50.145368 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.145379 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.145638 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.145854 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:50.146992 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.148504 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.151397 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.152983 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153114 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.153336 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.154463 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.154708 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":926,"threads_started":1}
I20260807 22:54:50.157436 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.141478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39575 (local address 127.0.0.1:34390)
0807 22:54:50.144903 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.144931 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.145074 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.146144 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.146153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.146183 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:50.146770 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.146780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.148670 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.148675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.151234 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.151248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.153509 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.153516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.154156 (+ 640us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.154219 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.154794 (+ 575us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.155288 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3258,"mutex_wait_us":488,"thread_start_us":2254,"threads_started":1}
I20260807 22:54:50.159962 28637 tablet_service.cc:1467] Processing CreateTablet for tablet 7940a1e2b8264409bf5ac9194e43acea (DEFAULT_TABLE table=test-table [id=a648e9a6402243dcb5d52f56393792e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:50.159962 28784 tablet_service.cc:1467] Processing CreateTablet for tablet 7940a1e2b8264409bf5ac9194e43acea (DEFAULT_TABLE table=test-table [id=a648e9a6402243dcb5d52f56393792e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:50.160938 28498 tablet_service.cc:1467] Processing CreateTablet for tablet 7940a1e2b8264409bf5ac9194e43acea (DEFAULT_TABLE table=test-table [id=a648e9a6402243dcb5d52f56393792e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:50.161475 28784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7940a1e2b8264409bf5ac9194e43acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:50.161489 28637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7940a1e2b8264409bf5ac9194e43acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:50.162195 28498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7940a1e2b8264409bf5ac9194e43acea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:50.176542 28876 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Bootstrap starting.
I20260807 22:54:50.177634 28874 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Bootstrap starting.
I20260807 22:54:50.180120 28875 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Bootstrap starting.
I20260807 22:54:50.181262 28876 tablet_bootstrap.cc:654] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:50.181427 28874 tablet_bootstrap.cc:654] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:50.182442 28874 log.cc:824] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:50.182717 28876 log.cc:824] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:50.184474 28875 tablet_bootstrap.cc:654] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:50.185464 28876 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: No bootstrap required, opened a new log
I20260807 22:54:50.185726 28876 ts_tablet_manager.cc:1397] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260807 22:54:50.185838 28874 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: No bootstrap required, opened a new log
I20260807 22:54:50.185824 28875 log.cc:824] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:50.186138 28874 ts_tablet_manager.cc:1397] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 22:54:50.189055 28875 tablet_bootstrap.cc:492] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: No bootstrap required, opened a new log
I20260807 22:54:50.189339 28875 ts_tablet_manager.cc:1397] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:54:50.193648 28876 raft_consensus.cc:348] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.194020 28876 raft_consensus.cc:374] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:50.194137 28876 raft_consensus.cc:729] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2342b9f0de464dda93c497876d937b17, State: Initialized, Role: FOLLOWER
I20260807 22:54:50.194743 28876 consensus_queue.cc:260] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.195817 28852 heartbeater.cc:503] Master 127.9.194.254:36487 was elected leader, sending a full tablet report...
I20260807 22:54:50.196481 28876 ts_tablet_manager.cc:1428] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260807 22:54:50.197911 28874 raft_consensus.cc:348] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.198354 28874 raft_consensus.cc:374] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:50.198470 28874 raft_consensus.cc:729] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f6e21e7e28429ab4cb4f53101fb851, State: Initialized, Role: FOLLOWER
I20260807 22:54:50.199118 28874 consensus_queue.cc:260] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.201334 28875 raft_consensus.cc:348] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.201817 28874 ts_tablet_manager.cc:1428] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Time spent starting tablet: real 0.016s user 0.018s sys 0.000s
I20260807 22:54:50.201884 28875 raft_consensus.cc:374] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:50.202013 28875 raft_consensus.cc:729] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ce4622b78041d7abfe5f66017bfaec, State: Initialized, Role: FOLLOWER
I20260807 22:54:50.202625 28875 consensus_queue.cc:260] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.203934 28708 heartbeater.cc:503] Master 127.9.194.254:36487 was elected leader, sending a full tablet report...
I20260807 22:54:50.204473 28875 ts_tablet_manager.cc:1428] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260807 22:54:50.236655 28568 tablet.cc:2369] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:54:50.286453 28709 tablet.cc:2369] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:54:50.322198 28853 tablet.cc:2369] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:54:50.515296 28882 raft_consensus.cc:482] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:50.515709 28882 raft_consensus.cc:504] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.517545 28882 leader_election.cc:288] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59f6e21e7e28429ab4cb4f53101fb851 (127.9.194.193:39575), 2342b9f0de464dda93c497876d937b17 (127.9.194.195:36195)
I20260807 22:54:50.525511 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.517939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:36195 (local address 127.9.194.194:59911)
0807 22:54:50.519051 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.519085 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.519266 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.519714 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.519718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.519747 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:50.520151 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.520166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.521302 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.521307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.523192 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.523204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.524747 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.524753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.524928 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.524963 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.525066 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.525204 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":693,"threads_started":1}
I20260807 22:54:50.525578 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.517987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:59911 (local address 127.9.194.195:36195)
0807 22:54:50.518194 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.518199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.519190 (+ 991us) server_negotiation.cc:407] Connection header received
0807 22:54:50.519471 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.519478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.519574 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.519672 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:50.520332 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.521166 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.523384 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.524524 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.524657 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.524712 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.525123 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.525278 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:54:50.526033 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.518064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:53889 (local address 127.9.194.193:39575)
0807 22:54:50.518295 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.518302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.519596 (+ 1294us) server_negotiation.cc:407] Connection header received
0807 22:54:50.519778 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.519785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.519929 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.520036 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:50.520660 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.521506 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.523993 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.524967 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.525130 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.525199 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.525643 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.525812 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:54:50.526245 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.518630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:39575 (local address 127.9.194.194:53889)
0807 22:54:50.519497 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:50.519514 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:50.519640 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:50.520057 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:50.520060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:50.520086 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:50.520503 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.520515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.521655 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.521659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.523833 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:50.523849 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.525213 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.525221 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:50.525430 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.525467 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:54:50.525580 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:54:50.525741 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":3,"thread_start_us":456,"threads_started":1}
I20260807 22:54:50.526737 28806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7940a1e2b8264409bf5ac9194e43acea" candidate_uuid: "88ce4622b78041d7abfe5f66017bfaec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342b9f0de464dda93c497876d937b17" is_pre_election: true
I20260807 22:54:50.527228 28806 raft_consensus.cc:2454] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ce4622b78041d7abfe5f66017bfaec in term 0.
I20260807 22:54:50.527316 28520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7940a1e2b8264409bf5ac9194e43acea" candidate_uuid: "88ce4622b78041d7abfe5f66017bfaec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f6e21e7e28429ab4cb4f53101fb851" is_pre_election: true
I20260807 22:54:50.527854 28520 raft_consensus.cc:2454] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ce4622b78041d7abfe5f66017bfaec in term 0.
I20260807 22:54:50.527943 28592 leader_election.cc:302] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2342b9f0de464dda93c497876d937b17, 88ce4622b78041d7abfe5f66017bfaec; no voters:
I20260807 22:54:50.528299 28882 raft_consensus.cc:2781] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:54:50.528448 28882 raft_consensus.cc:482] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:54:50.528570 28882 raft_consensus.cc:3037] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:50.530946 28882 raft_consensus.cc:504] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.531659 28882 leader_election.cc:288] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [CANDIDATE]: Term 1 election: Requested vote from peers 59f6e21e7e28429ab4cb4f53101fb851 (127.9.194.193:39575), 2342b9f0de464dda93c497876d937b17 (127.9.194.195:36195)
I20260807 22:54:50.532042 28520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7940a1e2b8264409bf5ac9194e43acea" candidate_uuid: "88ce4622b78041d7abfe5f66017bfaec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f6e21e7e28429ab4cb4f53101fb851"
I20260807 22:54:50.532186 28806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7940a1e2b8264409bf5ac9194e43acea" candidate_uuid: "88ce4622b78041d7abfe5f66017bfaec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342b9f0de464dda93c497876d937b17"
I20260807 22:54:50.532263 28520 raft_consensus.cc:3037] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:50.532392 28806 raft_consensus.cc:3037] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:50.534703 28520 raft_consensus.cc:2454] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ce4622b78041d7abfe5f66017bfaec in term 1.
I20260807 22:54:50.535164 28591 leader_election.cc:302] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59f6e21e7e28429ab4cb4f53101fb851, 88ce4622b78041d7abfe5f66017bfaec; no voters:
I20260807 22:54:50.535521 28882 raft_consensus.cc:2781] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:50.535622 28806 raft_consensus.cc:2454] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ce4622b78041d7abfe5f66017bfaec in term 1.
I20260807 22:54:50.536065 28882 raft_consensus.cc:686] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 1 LEADER]: Becoming Leader. State: Replica: 88ce4622b78041d7abfe5f66017bfaec, State: Running, Role: LEADER
I20260807 22:54:50.537025 28882 consensus_queue.cc:237] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } }
I20260807 22:54:50.542075 28364 catalog_manager.cc:5495] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ce4622b78041d7abfe5f66017bfaec (127.9.194.194). New cstate: current_term: 1 leader_uuid: "88ce4622b78041d7abfe5f66017bfaec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ce4622b78041d7abfe5f66017bfaec" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 } health_report { overall_health: UNKNOWN } } }
I20260807 22:54:50.607884 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.602245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53246 (local address 127.9.194.194:33155)
0807 22:54:50.602455 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.602463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.602486 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:54:50.602642 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.602646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.602724 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.602876 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:54:50.603290 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.604068 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.605596 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.606056 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.606190 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.606726 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:54:50.607237 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:54:50.607407 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.607525 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.607608 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:54:50.616101 28806 raft_consensus.cc:1261] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 1 FOLLOWER]: Refusing update from remote peer 88ce4622b78041d7abfe5f66017bfaec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:50.616151 28520 raft_consensus.cc:1261] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 1 FOLLOWER]: Refusing update from remote peer 88ce4622b78041d7abfe5f66017bfaec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:50.617072 28885 consensus_queue.cc:1035] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [LEADER]: Connected to new peer: Peer: permanent_uuid: "2342b9f0de464dda93c497876d937b17" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:50.617458 28882 consensus_queue.cc:1035] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f6e21e7e28429ab4cb4f53101fb851" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:50.629151 28888 mvcc.cc:205] Tried to move back new op lower bound from 7316042918353313792 to 7316042918044258304. Current Snapshot: MvccSnapshot[applied={T|T < 7316042918353313792}]
I20260807 22:54:50.630213 28889 mvcc.cc:205] Tried to move back new op lower bound from 7316042918353313792 to 7316042918044258304. Current Snapshot: MvccSnapshot[applied={T|T < 7316042918353313792}]
I20260807 22:54:50.642982 28363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:50.643302 28363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:50.647491 28363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75c42853da794a8d9a8c5e7445ec6c22: Sending DeleteTablet for 3 replicas of tablet 7940a1e2b8264409bf5ac9194e43acea
I20260807 22:54:50.648790 28784 tablet_service.cc:1514] Processing DeleteTablet for tablet 7940a1e2b8264409bf5ac9194e43acea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:50 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:45428
I20260807 22:54:50.648803 28498 tablet_service.cc:1514] Processing DeleteTablet for tablet 7940a1e2b8264409bf5ac9194e43acea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:50 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:34390
I20260807 22:54:50.648799 28637 tablet_service.cc:1514] Processing DeleteTablet for tablet 7940a1e2b8264409bf5ac9194e43acea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:50 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:53238
I20260807 22:54:50.655182 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28430
I20260807 22:54:50.661195 28900 tablet_replica.cc:330] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: stopping tablet replica
I20260807 22:54:50.661659 28900 raft_consensus.cc:2229] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:50.662151 28900 raft_consensus.cc:2258] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:50.665985 28900 ts_tablet_manager.cc:1905] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:50.667193 28898 tablet_replica.cc:330] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: stopping tablet replica
I20260807 22:54:50.667865 28899 tablet_replica.cc:330] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: stopping tablet replica
I20260807 22:54:50.668009 28898 raft_consensus.cc:2229] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:50.668291 28899 raft_consensus.cc:2229] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:54:50.668534 28898 raft_consensus.cc:2258] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:50.669305 28899 raft_consensus.cc:2258] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:50.670624 28898 ts_tablet_manager.cc:1905] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:54:50 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:54:50.677842 28899 ts_tablet_manager.cc:1905] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:50.678776 28898 ts_tablet_manager.cc:1918] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:50.678949 28898 log.cc:1196] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/7940a1e2b8264409bf5ac9194e43acea
I20260807 22:54:50.679535 28898 ts_tablet_manager.cc:1939] T 7940a1e2b8264409bf5ac9194e43acea P 59f6e21e7e28429ab4cb4f53101fb851: Deleting consensus metadata
I20260807 22:54:50.681376 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.656905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.9.194.193:39575)
0807 22:54:50.657085 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.657092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.657860 (+ 768us) server_negotiation.cc:407] Connection header received
0807 22:54:50.657933 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.657938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.658048 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.658154 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:50.661031 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.662202 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.665493 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.666124 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.666226 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.672924 (+ 6698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:50.672938 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:50.672959 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:50.673000 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:50.677502 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.678623 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.678632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.678635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.678812 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.679500 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.679507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.679512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.679990 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:50.680328 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.680749 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.680978 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:54:50.682371 28348 catalog_manager.cc:4841] TS 59f6e21e7e28429ab4cb4f53101fb851 (127.9.194.193:39575): tablet 7940a1e2b8264409bf5ac9194e43acea (table test-table [id=a648e9a6402243dcb5d52f56393792e2]) successfully deleted
I20260807 22:54:50.683909 28562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:50.685799 28899 ts_tablet_manager.cc:1918] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:50.686040 28899 log.cc:1196] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/7940a1e2b8264409bf5ac9194e43acea
I20260807 22:54:50.686527 28899 ts_tablet_manager.cc:1939] T 7940a1e2b8264409bf5ac9194e43acea P 88ce4622b78041d7abfe5f66017bfaec: Deleting consensus metadata
I20260807 22:54:50.688097 28350 catalog_manager.cc:4841] TS 88ce4622b78041d7abfe5f66017bfaec (127.9.194.194:33155): tablet 7940a1e2b8264409bf5ac9194e43acea (table test-table [id=a648e9a6402243dcb5d52f56393792e2]) successfully deleted
I20260807 22:54:50.689671 28900 ts_tablet_manager.cc:1918] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:50.689872 28900 log.cc:1196] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/7940a1e2b8264409bf5ac9194e43acea
I20260807 22:54:50.690376 28900 ts_tablet_manager.cc:1939] T 7940a1e2b8264409bf5ac9194e43acea P 2342b9f0de464dda93c497876d937b17: Deleting consensus metadata
I20260807 22:54:50.691866 28349 catalog_manager.cc:4841] TS 2342b9f0de464dda93c497876d937b17 (127.9.194.195:36195): tablet 7940a1e2b8264409bf5ac9194e43acea (table test-table [id=a648e9a6402243dcb5d52f56393792e2]) successfully deleted
I20260807 22:54:50.794902 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28430
I20260807 22:54:50.820505 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28573
Aug 07 22:54:50 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:54:50.838023 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.821427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.9.194.194:33155)
0807 22:54:50.821652 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.821658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.821681 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:54:50.821756 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.821762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.821850 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.821962 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:50.823271 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.824553 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.825925 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.826605 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.826755 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.830985 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:50.831005 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:50.831027 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:50.831069 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:50.835099 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.835816 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.835826 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.835829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.836007 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.836443 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.836449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.836452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.836881 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:50.837223 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.837379 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.837625 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:54:50.838920 28703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:50.955788 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28573
I20260807 22:54:50.976585 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28715
Aug 07 22:54:50 dist-test-slave-5k54 krb5kdc[28315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:54:50.990979 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:50.977370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.9.194.195:36195)
0807 22:54:50.977565 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:50.977570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:50.977588 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:54:50.977737 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:50.977742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:50.977816 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:50.977900 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:50.979186 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.979957 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.981149 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:50.981527 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:50.981619 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:50.985738 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:50.985750 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:50.985765 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:50.985797 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:50.988587 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.989193 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.989200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.989203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.989338 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:50.989655 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:50.989658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:50.989659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:50.989981 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:50.990186 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:50.990426 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 22:54:50.990602 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:54:50.991668 28847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:51.081758 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28715
I20260807 22:54:51.100988 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28329
I20260807 22:54:51.102157 28395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:51.216096 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28329
2026-08-07T22:54:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4792 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T22:54:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:54:51Z Disabled control of system clock
I20260807 22:54:51.310381 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44773
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:54:51.646867 28911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:51.647156 28911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:51.647318 28911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:51.654541 28911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:51.654644 28911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:51.654729 28911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:51.654834 28911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:51.654909 28911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:51.664026 28911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:44773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:51.665341 28911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:51.667191 28911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:51.679483 28919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:51.679570 28916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:51.679559 28917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:51.680245 28911 server_base.cc:1034] running on GCE node
I20260807 22:54:51.681025 28911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:51.682432 28911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:51.683660 28911 hybrid_clock.cc:648] HybridClock initialized: now 1786143291683619 us; error 47 us; skew 500 ppm
I20260807 22:54:51.684019 28911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:51.691784 28911 webserver.cc:460] Webserver started at http://127.9.194.254:43779/ using document root <none> and password file <none>
I20260807 22:54:51.692503 28911 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:51.692633 28911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:51.692953 28911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:51.696025 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "2e44fc8ee6954324be2ab43ac3b06894"
format_stamp: "Formatted at 2026-08-07 22:54:51 on dist-test-slave-5k54"
server_key: "34bf9e9539def5a82877a651d4d337dc"
server_key_iv: "98687e46fa4abfc9419055afc8920675"
server_key_version: "encryptionkey@0"
I20260807 22:54:51.696807 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "2e44fc8ee6954324be2ab43ac3b06894"
format_stamp: "Formatted at 2026-08-07 22:54:51 on dist-test-slave-5k54"
server_key: "34bf9e9539def5a82877a651d4d337dc"
server_key_iv: "98687e46fa4abfc9419055afc8920675"
server_key_version: "encryptionkey@0"
I20260807 22:54:51.701923 28911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:54:51.705353 28925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:51.706595 28911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:51.706781 28911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "2e44fc8ee6954324be2ab43ac3b06894"
format_stamp: "Formatted at 2026-08-07 22:54:51 on dist-test-slave-5k54"
server_key: "34bf9e9539def5a82877a651d4d337dc"
server_key_iv: "98687e46fa4abfc9419055afc8920675"
server_key_version: "encryptionkey@0"
I20260807 22:54:51.706997 28911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:51.719209 28911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:51.720006 28911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:51.720290 28911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:51.739080 28911 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:44773
I20260807 22:54:51.739114 28977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:44773 every 8 connection(s)
I20260807 22:54:51.740448 28911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:51.744187 28978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:51.745469 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 28911
I20260807 22:54:51.745662 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:54:51.746117 9995 external_mini_cluster.cc:1428] Setting key 1e95b4bf13f4df82025d8c7bfef91df6
I20260807 22:54:51.758062 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: Bootstrap starting.
I20260807 22:54:51.760051 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:51.747940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.9.194.254:44773)
0807 22:54:51.748950 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:51.748963 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:51.749006 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:54:51.750550 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:51.750571 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:51.751037 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:51.751362 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:51.752019 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:51.753270 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:51.755379 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:51.756114 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:51.756248 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:51.757677 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:51.757684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:51.757698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:51.758281 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:51.758359 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:51.758381 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:51.758482 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:51.758710 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 22:54:51.759033 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":391,"threads_started":1}
I20260807 22:54:51.761415 28978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:51.762458 28978 log.cc:824] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:51.765189 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: No bootstrap required, opened a new log
I20260807 22:54:51.772351 28978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } }
I20260807 22:54:51.772732 28978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:51.772837 28978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e44fc8ee6954324be2ab43ac3b06894, State: Initialized, Role: FOLLOWER
I20260807 22:54:51.773409 28978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } }
I20260807 22:54:51.773658 28978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:51.773798 28978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:51.773972 28978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:51.776106 28978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } }
I20260807 22:54:51.776597 28978 leader_election.cc:302] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e44fc8ee6954324be2ab43ac3b06894; no voters:
I20260807 22:54:51.777179 28978 leader_election.cc:288] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:54:51.777310 28985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:51.778597 28985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [term 1 LEADER]: Becoming Leader. State: Replica: 2e44fc8ee6954324be2ab43ac3b06894, State: Running, Role: LEADER
I20260807 22:54:51.779291 28978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:51.779091 28985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } }
I20260807 22:54:51.782321 28986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e44fc8ee6954324be2ab43ac3b06894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } } }
I20260807 22:54:51.782302 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e44fc8ee6954324be2ab43ac3b06894. Latest consensus state: current_term: 1 leader_uuid: "2e44fc8ee6954324be2ab43ac3b06894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e44fc8ee6954324be2ab43ac3b06894" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 44773 } } }
I20260807 22:54:51.782789 28986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:51.782800 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:51.790499 28991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:51.795400 28991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:51.803241 28991 catalog_manager.cc:1347] Generated new cluster ID: c5585af6d4ac47af88d16d5aaa65a108
I20260807 22:54:51.803356 28991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:51.813850 28991 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:54:51.815646 28991 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:51.825803 28991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: Generated new TSK 0
I20260807 22:54:51.826934 28991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:54:51.835628 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--builtin_ntp_servers=127.9.194.212:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:54:52.177045 29006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:52.177331 29006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:52.177421 29006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:52.184741 29006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:52.184851 29006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:52.184957 29006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:52.194092 29006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:52.195338 29006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:52.197137 29006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:52.210266 29011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:52.210165 29012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:52.210228 29014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:52.210677 29006 server_base.cc:1034] running on GCE node
I20260807 22:54:52.211835 29006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:52.212775 29006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:52.214005 29006 hybrid_clock.cc:648] HybridClock initialized: now 1786143292213948 us; error 47 us; skew 500 ppm
I20260807 22:54:52.214344 29006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:52.216672 29006 webserver.cc:460] Webserver started at http://127.9.194.193:36889/ using document root <none> and password file <none>
I20260807 22:54:52.217389 29006 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:52.217482 29006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:52.217839 29006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:52.220805 29006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "7a158cd902234c0ca81e8667b134deaa"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "ea6e6a2edb5e9df93bf1617521199b1b"
server_key_iv: "6cb3d22dc992295f49552c435dfd30a5"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.221617 29006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "7a158cd902234c0ca81e8667b134deaa"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "ea6e6a2edb5e9df93bf1617521199b1b"
server_key_iv: "6cb3d22dc992295f49552c435dfd30a5"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.226883 29006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:54:52.230693 29020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.232179 29006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:52.232357 29006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "7a158cd902234c0ca81e8667b134deaa"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "ea6e6a2edb5e9df93bf1617521199b1b"
server_key_iv: "6cb3d22dc992295f49552c435dfd30a5"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.232605 29006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:52.283318 29006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:52.284226 29006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:52.284536 29006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:52.285532 29006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:52.287026 29006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:52.287154 29006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.287297 29006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:52.287387 29006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.320873 29006 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:45901
I20260807 22:54:52.320959 29144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:45901 every 8 connection(s)
I20260807 22:54:52.322167 29006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:54:52.325758 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29006
I20260807 22:54:52.325953 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:54:52.326440 9995 external_mini_cluster.cc:1428] Setting key c0444004f174b7d311db4b5f0b33b131
I20260807 22:54:52.329181 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--builtin_ntp_servers=127.9.194.212:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:54:52.335070 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:52.324770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:60625 (local address 127.9.194.254:44773)
0807 22:54:52.325351 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:52.325358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:52.326700 (+ 1342us) server_negotiation.cc:407] Connection header received
0807 22:54:52.328591 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:52.328596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:52.328683 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:52.328803 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:52.329982 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.330700 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.332313 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.332714 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.332848 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:52.334227 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:52.334232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:52.334235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:52.334253 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:52.334292 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:52.334311 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:52.334405 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:52.334616 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 22:54:52.334714 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":313,"threads_started":1}
I20260807 22:54:52.336187 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:52.325456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44773 (local address 127.9.194.193:60625)
0807 22:54:52.326527 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:52.326578 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:52.328308 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:52.329032 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:52.329040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:52.329125 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:52.329781 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:52.329799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.330854 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.330859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:52.332153 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:52.332165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.332960 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.332968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:52.333180 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:52.333922 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:54:52.333924 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:52.333962 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:54:52.334033 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:52.334404 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:52.334419 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:52.334429 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:54:52.334605 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 22:54:52.335002 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":351,"threads_started":1}
I20260807 22:54:52.337641 29145 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44773
I20260807 22:54:52.337971 29145 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:52.338855 29145 heartbeater.cc:511] Master 127.9.194.254:44773 requested a full tablet report, sending...
I20260807 22:54:52.341344 28942 ts_manager.cc:194] Registered new tserver with Master: 7a158cd902234c0ca81e8667b134deaa (127.9.194.193:45901)
I20260807 22:54:52.342900 28942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.193:60625
W20260807 22:54:52.663789 29149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:52.664132 29149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:52.664299 29149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:52.671674 29149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:52.671813 29149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:52.671993 29149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:54:52.681433 29149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:52.682751 29149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:52.684579 29149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:52.697965 29159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:52.698046 29157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:52.697988 29156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:52.699239 29149 server_base.cc:1034] running on GCE node
I20260807 22:54:52.700030 29149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:52.701037 29149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:52.702281 29149 hybrid_clock.cc:648] HybridClock initialized: now 1786143292702227 us; error 60 us; skew 500 ppm
I20260807 22:54:52.702608 29149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:52.704950 29149 webserver.cc:460] Webserver started at http://127.9.194.194:46243/ using document root <none> and password file <none>
I20260807 22:54:52.705654 29149 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:52.705778 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:52.706108 29149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:52.709059 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "d751ce29cfc24709b9566c3a895b463f"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "9c2eb48cfb834b662827ce8cc8f9378b"
server_key_iv: "0811e82625839a148cdd3926aec2aae6"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.709997 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "d751ce29cfc24709b9566c3a895b463f"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "9c2eb48cfb834b662827ce8cc8f9378b"
server_key_iv: "0811e82625839a148cdd3926aec2aae6"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.714972 29149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:54:52.718461 29166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.719892 29149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:52.720067 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "d751ce29cfc24709b9566c3a895b463f"
format_stamp: "Formatted at 2026-08-07 22:54:52 on dist-test-slave-5k54"
server_key: "9c2eb48cfb834b662827ce8cc8f9378b"
server_key_iv: "0811e82625839a148cdd3926aec2aae6"
server_key_version: "encryptionkey@0"
I20260807 22:54:52.720253 29149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:52.754364 29149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:52.755188 29149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:52.755519 29149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:52.756471 29149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:52.757866 29149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:52.757951 29149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.758030 29149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:52.758083 29149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:52.791658 29149 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:41695
I20260807 22:54:52.791747 29290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:41695 every 8 connection(s)
I20260807 22:54:52.792974 29149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:54:52.801668 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29149
I20260807 22:54:52.801865 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:54:52.802462 9995 external_mini_cluster.cc:1428] Setting key b6049ea6d1a9614c020de4a6e2d31da1
I20260807 22:54:52.805898 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--builtin_ntp_servers=127.9.194.212:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:54:52.806210 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:52.795578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:56527 (local address 127.9.194.254:44773)
0807 22:54:52.795751 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:52.795755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:52.797044 (+ 1289us) server_negotiation.cc:407] Connection header received
0807 22:54:52.798891 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:52.798897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:52.798988 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:52.799098 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:52.800179 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.800904 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.803184 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.803740 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.803864 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:52.805415 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:52.805419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:52.805423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:52.805439 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:52.805470 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:52.805482 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:52.805552 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:52.805808 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 22:54:52.805881 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:54:52.807669 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:52.796004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44773 (local address 127.9.194.194:56527)
0807 22:54:52.796876 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:52.796926 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:52.798632 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:52.799316 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:52.799324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:52.799411 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:52.800019 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:52.800035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.801045 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.801051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:52.803025 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:52.803040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:52.803982 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:52.803992 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:52.804192 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:52.805059 (+ 867us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:54:52.805062 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:52.805117 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:54:52.805215 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:52.805582 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:52.805596 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:52.805607 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:54:52.805803 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 22:54:52.806235 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":280,"threads_started":1}
I20260807 22:54:52.809046 29291 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44773
I20260807 22:54:52.809329 29291 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:52.810084 29291 heartbeater.cc:511] Master 127.9.194.254:44773 requested a full tablet report, sending...
I20260807 22:54:52.811475 28942 ts_manager.cc:194] Registered new tserver with Master: d751ce29cfc24709b9566c3a895b463f (127.9.194.194:41695)
I20260807 22:54:52.812284 28942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.194:56527
W20260807 22:54:53.145314 29295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:53.145592 29295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:53.145763 29295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:53.152930 29295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:53.153026 29295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:53.153191 29295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:54:53.162099 29295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:53.163447 29295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:53.165869 29295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:53.179814 29300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:53.179827 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:53.179737 29303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:53.184837 29295 server_base.cc:1034] running on GCE node
I20260807 22:54:53.185386 29295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:53.186286 29295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:53.187521 29295 hybrid_clock.cc:648] HybridClock initialized: now 1786143293187473 us; error 40 us; skew 500 ppm
I20260807 22:54:53.187829 29295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:53.190168 29295 webserver.cc:460] Webserver started at http://127.9.194.195:36819/ using document root <none> and password file <none>
I20260807 22:54:53.190847 29295 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:53.190973 29295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:53.191299 29295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:53.194164 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "3e9e4b3cbf364f099eb3fdb954570f09"
format_stamp: "Formatted at 2026-08-07 22:54:53 on dist-test-slave-5k54"
server_key: "629ed121ed0a839bddab8f4b76643993"
server_key_iv: "c1339aea6772c4a8f3196e4a939d49cd"
server_key_version: "encryptionkey@0"
I20260807 22:54:53.195013 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "3e9e4b3cbf364f099eb3fdb954570f09"
format_stamp: "Formatted at 2026-08-07 22:54:53 on dist-test-slave-5k54"
server_key: "629ed121ed0a839bddab8f4b76643993"
server_key_iv: "c1339aea6772c4a8f3196e4a939d49cd"
server_key_version: "encryptionkey@0"
I20260807 22:54:53.200032 29295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:54:53.203488 29312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:53.204834 29295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:54:53.205026 29295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "3e9e4b3cbf364f099eb3fdb954570f09"
format_stamp: "Formatted at 2026-08-07 22:54:53 on dist-test-slave-5k54"
server_key: "629ed121ed0a839bddab8f4b76643993"
server_key_iv: "c1339aea6772c4a8f3196e4a939d49cd"
server_key_version: "encryptionkey@0"
I20260807 22:54:53.205238 29295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:53.225570 29295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:53.226424 29295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:53.226733 29295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:53.227744 29295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:53.229249 29295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:53.229359 29295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:53.229485 29295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:53.229568 29295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:53.263290 29295 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:37929
I20260807 22:54:53.263417 29431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:37929 every 8 connection(s)
I20260807 22:54:53.264608 29295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:54:53.266088 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29295
I20260807 22:54:53.266276 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:54:53.266804 9995 external_mini_cluster.cc:1428] Setting key 48b4fb0bc720a9b1f781a5615c4e13b9
I20260807 22:54:53.277880 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.267310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:40687 (local address 127.9.194.254:44773)
0807 22:54:53.267593 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.267599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.269390 (+ 1791us) server_negotiation.cc:407] Connection header received
0807 22:54:53.271333 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.271336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.271401 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.271524 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:53.272553 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.273498 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.275125 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.275600 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.275725 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.277083 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:53.277089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:53.277094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:53.277119 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:53.277149 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:53.277160 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:53.277251 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.277456 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.277550 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:54:53.278811 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.268092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:44773 (local address 127.9.194.195:40687)
0807 22:54:53.269223 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.269277 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.271080 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.271706 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.271714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.271811 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:53.272360 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.272412 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.273641 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.273651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.274907 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.274919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.275822 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.275830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.276073 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.276779 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:54:53.276782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:53.276818 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:54:53.276895 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:53.277250 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:53.277264 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:53.277274 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.277453 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.277827 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":339,"threads_started":1}
I20260807 22:54:53.280175 29432 heartbeater.cc:347] Connected to a master server at 127.9.194.254:44773
I20260807 22:54:53.280530 29432 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:53.281349 29432 heartbeater.cc:511] Master 127.9.194.254:44773 requested a full tablet report, sending...
I20260807 22:54:53.282799 28942 ts_manager.cc:194] Registered new tserver with Master: 3e9e4b3cbf364f099eb3fdb954570f09 (127.9.194.195:37929)
I20260807 22:54:53.283618 28942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.195:40687
I20260807 22:54:53.294143 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:54:53.304611 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.298558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.9.194.254:44773)
0807 22:54:53.298838 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.298847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.299182 (+ 335us) server_negotiation.cc:407] Connection header received
0807 22:54:53.299384 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.299389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.299545 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.299653 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:53.300148 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.300836 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.302220 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.302644 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.302777 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.303877 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:53.303883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:53.303887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:53.303911 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:53.303943 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:53.303955 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:53.304055 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.304212 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.304303 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:54:53.308756 28942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:54:53.315936 28942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:54:53.345733 29145 heartbeater.cc:503] Master 127.9.194.254:44773 was elected leader, sending a full tablet report...
I20260807 22:54:53.355947 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.328605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:37929 (local address 127.0.0.1:47690)
0807 22:54:53.342740 (+ 14135us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.342798 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.343088 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.345195 (+ 2107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.345207 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.345251 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:53.346792 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.346807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.349590 (+ 2783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.349599 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.353037 (+ 3438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.353050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.354858 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.354865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.355085 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.355142 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.355291 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.355542 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13765,"mutex_wait_us":103,"thread_start_us":15562,"threads_started":1}
I20260807 22:54:53.356308 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.329359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39628)
0807 22:54:53.344468 (+ 15109us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.344488 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.344609 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.345494 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.345502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.345541 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:53.346911 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.346929 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.350428 (+ 3499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.350439 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.353871 (+ 3432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.353881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.355330 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.355341 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.355564 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.355600 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.355708 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.355847 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14970,"mutex_wait_us":58,"thread_start_us":12844,"threads_started":1}
I20260807 22:54:53.356779 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.329505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39628 (local address 127.9.194.193:45901)
0807 22:54:53.341961 (+ 12456us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.341983 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.344749 (+ 2766us) server_negotiation.cc:407] Connection header received
0807 22:54:53.344894 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.344906 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.345223 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.345656 (+ 433us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:53.348150 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.350214 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.354045 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.355115 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.355267 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.355560 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.355760 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.355953 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12190,"thread_start_us":13789,"threads_started":1}
I20260807 22:54:53.357092 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.328816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.9.194.195:37929)
0807 22:54:53.344057 (+ 15241us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.344066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.344103 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:54:53.344213 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.344221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.344437 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.344681 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:53.347084 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.349339 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.353240 (+ 3901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.354657 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.354788 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.355062 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.356550 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.356722 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15046,"thread_start_us":16117,"threads_started":1}
I20260807 22:54:53.359822 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.344605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41695 (local address 127.0.0.1:33208)
0807 22:54:53.351785 (+ 7180us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.351815 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.351941 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.352894 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.352899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.352925 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:53.353368 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.353379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.355173 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.355176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.357491 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.357510 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.359062 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.359075 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.359252 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.359283 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.359385 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.359534 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7047,"thread_start_us":6588,"threads_started":1}
I20260807 22:54:53.359931 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.344960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.9.194.194:41695)
0807 22:54:53.351564 (+ 6604us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.351575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.352081 (+ 506us) server_negotiation.cc:407] Connection header received
0807 22:54:53.352190 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.352197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.352457 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.352635 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:53.353554 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.355021 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.357673 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.358850 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.359010 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.359279 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.359443 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.359600 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6403,"thread_start_us":6324,"threads_started":1}
I20260807 22:54:53.361100 29364 tablet_service.cc:1467] Processing CreateTablet for tablet 006db1693f084fb387d02bb630f3aea8 (DEFAULT_TABLE table=test-table [id=dd99fc1fc36f4277ac95d84a6217d89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:53.360985 29071 tablet_service.cc:1467] Processing CreateTablet for tablet 006db1693f084fb387d02bb630f3aea8 (DEFAULT_TABLE table=test-table [id=dd99fc1fc36f4277ac95d84a6217d89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:53.362329 29364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006db1693f084fb387d02bb630f3aea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:53.362414 29071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006db1693f084fb387d02bb630f3aea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:53.363554 29220 tablet_service.cc:1467] Processing CreateTablet for tablet 006db1693f084fb387d02bb630f3aea8 (DEFAULT_TABLE table=test-table [id=dd99fc1fc36f4277ac95d84a6217d89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:53.364884 29220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006db1693f084fb387d02bb630f3aea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:53.374665 29450 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Bootstrap starting.
I20260807 22:54:53.377436 29451 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Bootstrap starting.
I20260807 22:54:53.378669 29450 tablet_bootstrap.cc:654] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:53.380263 29450 log.cc:824] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:53.380295 29453 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Bootstrap starting.
I20260807 22:54:53.381369 29451 tablet_bootstrap.cc:654] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:53.383033 29451 log.cc:824] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:53.383301 29453 tablet_bootstrap.cc:654] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:53.384485 29453 log.cc:824] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:53.386240 29451 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: No bootstrap required, opened a new log
I20260807 22:54:53.386365 29450 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: No bootstrap required, opened a new log
I20260807 22:54:53.386536 29451 ts_tablet_manager.cc:1397] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:54:53.386643 29450 ts_tablet_manager.cc:1397] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 22:54:53.387591 29453 tablet_bootstrap.cc:492] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: No bootstrap required, opened a new log
I20260807 22:54:53.387888 29453 ts_tablet_manager.cc:1397] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 22:54:53.393586 29451 raft_consensus.cc:348] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.393956 29451 raft_consensus.cc:374] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:53.394109 29451 raft_consensus.cc:729] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a158cd902234c0ca81e8667b134deaa, State: Initialized, Role: FOLLOWER
I20260807 22:54:53.394708 29451 consensus_queue.cc:260] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.396497 29451 ts_tablet_manager.cc:1428] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260807 22:54:53.397248 29450 raft_consensus.cc:348] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.397680 29450 raft_consensus.cc:374] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:53.397782 29450 raft_consensus.cc:729] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e4b3cbf364f099eb3fdb954570f09, State: Initialized, Role: FOLLOWER
I20260807 22:54:53.398389 29450 consensus_queue.cc:260] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.399555 29432 heartbeater.cc:503] Master 127.9.194.254:44773 was elected leader, sending a full tablet report...
I20260807 22:54:53.400321 29450 ts_tablet_manager.cc:1428] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 22:54:53.400346 29453 raft_consensus.cc:348] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.400771 29453 raft_consensus.cc:374] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:53.400899 29453 raft_consensus.cc:729] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d751ce29cfc24709b9566c3a895b463f, State: Initialized, Role: FOLLOWER
I20260807 22:54:53.401473 29453 consensus_queue.cc:260] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.403244 29291 heartbeater.cc:503] Master 127.9.194.254:44773 was elected leader, sending a full tablet report...
I20260807 22:54:53.404033 29453 ts_tablet_manager.cc:1428] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260807 22:54:53.516392 29433 tablet.cc:2369] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:54:53.545065 29292 tablet.cc:2369] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:54:53.574688 29146 tablet.cc:2369] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:54:53.670814 29458 raft_consensus.cc:482] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:53.671247 29458 raft_consensus.cc:504] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.672995 29458 leader_election.cc:288] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e4b3cbf364f099eb3fdb954570f09 (127.9.194.195:37929), d751ce29cfc24709b9566c3a895b463f (127.9.194.194:41695)
I20260807 22:54:53.680332 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.674487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:52295 (local address 127.9.194.194:41695)
0807 22:54:53.674688 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.674694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.674839 (+ 145us) server_negotiation.cc:407] Connection header received
0807 22:54:53.674975 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.674982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.675085 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.675181 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:53.675813 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.676534 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.678531 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.679329 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.679506 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.679581 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.679927 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.680062 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:54:53.680343 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.673524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:52275 (local address 127.9.194.195:37929)
0807 22:54:53.673745 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.673751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.674270 (+ 519us) server_negotiation.cc:407] Connection header received
0807 22:54:53.674528 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.674535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.674620 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.674688 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:53.675560 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.676368 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.678584 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.679414 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.679628 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.679713 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.680009 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.680133 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:54:53.680459 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.674157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:41695 (local address 127.9.194.193:52295)
0807 22:54:53.674737 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.674750 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.674855 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.675208 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.675211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.675232 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:53.675658 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.675672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.676683 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.676686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.678333 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.678345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.679544 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.679550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.679731 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.679805 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.679885 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.680022 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":53,"thread_start_us":308,"threads_started":1}
I20260807 22:54:53.680518 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.673448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:37929 (local address 127.9.194.193:52275)
0807 22:54:53.674122 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.674144 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:53.674321 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:53.674925 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:53.674929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:53.674958 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:53.675369 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.675381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.676505 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.676510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.678340 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:53.678353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.679621 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.679630 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:53.679826 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.679863 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:54:53.679964 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:54:53.680082 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":111,"thread_start_us":337,"threads_started":1}
I20260807 22:54:53.681507 29386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006db1693f084fb387d02bb630f3aea8" candidate_uuid: "7a158cd902234c0ca81e8667b134deaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" is_pre_election: true
I20260807 22:54:53.681509 29243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006db1693f084fb387d02bb630f3aea8" candidate_uuid: "7a158cd902234c0ca81e8667b134deaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d751ce29cfc24709b9566c3a895b463f" is_pre_election: true
I20260807 22:54:53.682015 29386 raft_consensus.cc:2454] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a158cd902234c0ca81e8667b134deaa in term 0.
I20260807 22:54:53.682021 29243 raft_consensus.cc:2454] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a158cd902234c0ca81e8667b134deaa in term 0.
I20260807 22:54:53.682670 29021 leader_election.cc:302] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a158cd902234c0ca81e8667b134deaa, d751ce29cfc24709b9566c3a895b463f; no voters:
I20260807 22:54:53.683202 29458 raft_consensus.cc:2781] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:54:53.683367 29458 raft_consensus.cc:482] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:54:53.683501 29458 raft_consensus.cc:3037] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:53.685638 29458 raft_consensus.cc:504] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.686224 29458 leader_election.cc:288] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e4b3cbf364f099eb3fdb954570f09 (127.9.194.195:37929), d751ce29cfc24709b9566c3a895b463f (127.9.194.194:41695)
I20260807 22:54:53.686612 29386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006db1693f084fb387d02bb630f3aea8" candidate_uuid: "7a158cd902234c0ca81e8667b134deaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e4b3cbf364f099eb3fdb954570f09"
I20260807 22:54:53.686720 29243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006db1693f084fb387d02bb630f3aea8" candidate_uuid: "7a158cd902234c0ca81e8667b134deaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d751ce29cfc24709b9566c3a895b463f"
I20260807 22:54:53.686810 29386 raft_consensus.cc:3037] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:53.686959 29243 raft_consensus.cc:3037] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:53.689347 29386 raft_consensus.cc:2454] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a158cd902234c0ca81e8667b134deaa in term 1.
I20260807 22:54:53.689364 29243 raft_consensus.cc:2454] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a158cd902234c0ca81e8667b134deaa in term 1.
I20260807 22:54:53.689733 29021 leader_election.cc:302] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e4b3cbf364f099eb3fdb954570f09, 7a158cd902234c0ca81e8667b134deaa; no voters:
I20260807 22:54:53.690021 29458 raft_consensus.cc:2781] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:53.690471 29458 raft_consensus.cc:686] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [term 1 LEADER]: Becoming Leader. State: Replica: 7a158cd902234c0ca81e8667b134deaa, State: Running, Role: LEADER
I20260807 22:54:53.690948 29458 consensus_queue.cc:237] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } }
I20260807 22:54:53.695924 28941 catalog_manager.cc:5495] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a158cd902234c0ca81e8667b134deaa (127.9.194.193). New cstate: current_term: 1 leader_uuid: "7a158cd902234c0ca81e8667b134deaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a158cd902234c0ca81e8667b134deaa" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 45901 } health_report { overall_health: HEALTHY } } }
I20260807 22:54:53.716486 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.711557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.9.194.193:45901)
0807 22:54:53.711733 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.711737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.711756 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:54:53.711896 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.711901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.711999 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.712128 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:54:53.712453 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.713156 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.714535 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.714934 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.715031 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.715480 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:54:53.716010 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:54:53.716122 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.716225 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.716286 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:54:53.724684 29243 raft_consensus.cc:1261] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 1 FOLLOWER]: Refusing update from remote peer 7a158cd902234c0ca81e8667b134deaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:53.724745 29386 raft_consensus.cc:1261] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 1 FOLLOWER]: Refusing update from remote peer 7a158cd902234c0ca81e8667b134deaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:53.725651 29465 consensus_queue.cc:1035] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d751ce29cfc24709b9566c3a895b463f" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:53.726051 29458 consensus_queue.cc:1035] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e4b3cbf364f099eb3fdb954570f09" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:53.738054 29468 mvcc.cc:205] Tried to move back new op lower bound from 7316042931086241792 to 7316042930961723392. Current Snapshot: MvccSnapshot[applied={T|T < 7316042931086241792}]
I20260807 22:54:53.753451 28942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:53.753726 28942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:53.757266 28942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e44fc8ee6954324be2ab43ac3b06894: Sending DeleteTablet for 3 replicas of tablet 006db1693f084fb387d02bb630f3aea8
I20260807 22:54:53.758680 29364 tablet_service.cc:1514] Processing DeleteTablet for tablet 006db1693f084fb387d02bb630f3aea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:53 UTC) from {username='slave'} at 127.0.0.1:47690
I20260807 22:54:53.758684 29071 tablet_service.cc:1514] Processing DeleteTablet for tablet 006db1693f084fb387d02bb630f3aea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:53 UTC) from {username='slave'} at 127.0.0.1:39628
I20260807 22:54:53.759441 29220 tablet_service.cc:1514] Processing DeleteTablet for tablet 006db1693f084fb387d02bb630f3aea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:53 UTC) from {username='slave'} at 127.0.0.1:33208
I20260807 22:54:53.767283 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29006
I20260807 22:54:53.783308 29481 tablet_replica.cc:330] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: stopping tablet replica
I20260807 22:54:53.783972 29481 raft_consensus.cc:2229] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:53.784636 29481 raft_consensus.cc:2258] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:53.785609 29479 tablet_replica.cc:330] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: stopping tablet replica
I20260807 22:54:53.786118 29479 raft_consensus.cc:2229] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:53.786716 29479 raft_consensus.cc:2258] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:53.787938 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.768662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.9.194.193:45901)
0807 22:54:53.768856 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.768862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.778896 (+ 10034us) server_negotiation.cc:407] Connection header received
0807 22:54:53.779378 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.779386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.779570 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.779650 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:53.780353 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.781610 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.783657 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.784372 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.784467 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.786070 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:53.786089 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:53.786110 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:53.786912 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:53.786984 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:53.787010 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:53.787088 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.787339 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.787482 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:54:53.789103 29140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:53.789637 29481 ts_tablet_manager.cc:1905] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:53.797461 29479 ts_tablet_manager.cc:1905] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:53.802772 29481 ts_tablet_manager.cc:1918] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:53.803036 29481 log.cc:1196] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/006db1693f084fb387d02bb630f3aea8
I20260807 22:54:53.803684 29481 ts_tablet_manager.cc:1939] T 006db1693f084fb387d02bb630f3aea8 P d751ce29cfc24709b9566c3a895b463f: Deleting consensus metadata
I20260807 22:54:53.804463 29479 ts_tablet_manager.cc:1918] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:53.804625 29479 log.cc:1196] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/006db1693f084fb387d02bb630f3aea8
I20260807 22:54:53.805150 29479 ts_tablet_manager.cc:1939] T 006db1693f084fb387d02bb630f3aea8 P 3e9e4b3cbf364f099eb3fdb954570f09: Deleting consensus metadata
I20260807 22:54:53.805954 28926 catalog_manager.cc:4841] TS d751ce29cfc24709b9566c3a895b463f (127.9.194.194:41695): tablet 006db1693f084fb387d02bb630f3aea8 (table test-table [id=dd99fc1fc36f4277ac95d84a6217d89b]) successfully deleted
I20260807 22:54:53.807538 28926 catalog_manager.cc:4841] TS 3e9e4b3cbf364f099eb3fdb954570f09 (127.9.194.195:37929): tablet 006db1693f084fb387d02bb630f3aea8 (table test-table [id=dd99fc1fc36f4277ac95d84a6217d89b]) successfully deleted
I20260807 22:54:53.909406 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29006
I20260807 22:54:53.909554 29480 tablet_replica.cc:330] T 006db1693f084fb387d02bb630f3aea8 P 7a158cd902234c0ca81e8667b134deaa: stopping tablet replica
W20260807 22:54:53.930997 28927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:45901 (error 108)
I20260807 22:54:53.931255 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29149
I20260807 22:54:53.933147 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.932291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39662)
0807 22:54:53.932798 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.932939 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":284,"threads_started":1}
W20260807 22:54:53.933378 28927 catalog_manager.cc:4584] TS 7a158cd902234c0ca81e8667b134deaa (127.9.194.193:45901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 006db1693f084fb387d02bb630f3aea8: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
I20260807 22:54:53.938536 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.932263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.9.194.194:41695)
0807 22:54:53.932450 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:53.932455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:53.932598 (+ 143us) server_negotiation.cc:407] Connection header received
0807 22:54:53.932792 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:53.932798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:53.932918 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:53.933029 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:53.933505 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.934289 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.935731 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:53.936131 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:53.936227 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:53.937191 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:53.937200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:53.937213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:53.937800 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:53.937862 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:53.937882 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:53.937967 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:53.938074 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 22:54:53.938157 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:54:53.939167 29286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:53.986948 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.986396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39674)
0807 22:54:53.986649 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.986765 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 22:54:53.988169 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:53.987718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39690)
0807 22:54:53.987894 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:53.988003 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 22:54:54.028477 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29149
I20260807 22:54:54.047679 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29295
I20260807 22:54:54.048790 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:54.048255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39706)
0807 22:54:54.048501 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:54.048601 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 22:54:54.049805 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:54.049493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39718)
0807 22:54:54.049630 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:54.049690 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:54:54.054566 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:54.048577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.9.194.195:37929)
0807 22:54:54.048736 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:54.048741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:54.048758 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:54:54.048817 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:54.048822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:54.048924 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:54.049052 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:54.049527 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:54.050344 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:54.051729 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:54.052151 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:54.052245 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:54.053234 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:54.053243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:54.053258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:54.053843 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:54.053889 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:54.053909 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:54.054012 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:54.054118 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 22:54:54.054204 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:54:54.055228 29427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:54.130668 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:54.130126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39720)
0807 22:54:54.130380 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:54.130499 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 22:54:54.131798 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:54.131380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:45901 (local address 127.0.0.1:39726)
0807 22:54:54.131605 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:54.131673 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:45901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 22:54:54.144452 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29295
I20260807 22:54:54.163050 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 28911
I20260807 22:54:54.164176 28973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:54.275950 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 28911
2026-08-07T22:54:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3053 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:54:54 dist-test-slave-5k54 krb5kdc[29492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:54:54 dist-test-slave-5k54 krb5kdc[29492](info): set up 2 sockets
Aug 07 22:54:54 dist-test-slave-5k54 krb5kdc[29492](info): commencing operation
krb5kdc: starting...
W20260807 22:54:55.440464 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:54:55 dist-test-slave-5k54 krb5kdc[29492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:54:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:54:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:54:55.755193 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:32995
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:32995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:56.092796 29506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:56.093099 29506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:56.093277 29506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:56.100580 29506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:56.100701 29506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:56.100785 29506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:56.100872 29506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:56.100953 29506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:56.111501 29506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:32995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:32995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:56.112874 29506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:56.114189 29506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:56.125831 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:56.125809 29516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:56.126050 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:56.126749 29506 server_base.cc:1034] running on GCE node
I20260807 22:54:56.127552 29506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:56.128885 29506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:56.130097 29506 hybrid_clock.cc:648] HybridClock initialized: now 1786143296130038 us; error 50 us; skew 500 ppm
I20260807 22:54:56.130463 29506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:56 dist-test-slave-5k54 krb5kdc[29492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:56.133646 29506 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:54:56.135098 29506 webserver.cc:460] Webserver started at http://127.9.194.254:39621/ using document root <none> and password file <none>
I20260807 22:54:56.135869 29506 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:56.136013 29506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:56.136317 29506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:56.139259 29506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "c237efd2e4ce438ca6f3b13182e5bce8"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "af296531d6ebc9f0878fe527279e6865"
server_key_iv: "797218a668fc58e132f06f15ea28c060"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.140317 29506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "c237efd2e4ce438ca6f3b13182e5bce8"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "af296531d6ebc9f0878fe527279e6865"
server_key_iv: "797218a668fc58e132f06f15ea28c060"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.145803 29506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:54:56.150089 29523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:56.151692 29506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:54:56.151885 29506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "c237efd2e4ce438ca6f3b13182e5bce8"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "af296531d6ebc9f0878fe527279e6865"
server_key_iv: "797218a668fc58e132f06f15ea28c060"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.152135 29506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:56.163919 29506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:56.168020 29506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:56.168303 29506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:56.188266 29506 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:32995
I20260807 22:54:56.188283 29577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:32995 every 8 connection(s)
I20260807 22:54:56.189726 29506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:56.193693 29578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:56.193789 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29506
I20260807 22:54:56.194008 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:54:56.194597 9995 external_mini_cluster.cc:1428] Setting key 85034f1bfcc1e3daada5cf0d0db4424f
I20260807 22:54:56.208725 29578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: Bootstrap starting.
Aug 07 22:54:56 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:56.212556 29578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:56.213627 29578 log.cc:824] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:56.216605 29578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: No bootstrap required, opened a new log
I20260807 22:54:56.218314 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:56.196602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.9.194.254:32995)
0807 22:54:56.197463 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:56.197479 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:56.197528 (+ 49us) server_negotiation.cc:407] Connection header received
0807 22:54:56.199187 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:56.199216 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:56.199748 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:56.200202 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:56.201571 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.203603 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.204951 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.206078 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.206266 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:56.210926 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:56.210934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:56.210967 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:56.211028 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:56.214174 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:56.214946 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:56.214952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:56.214958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:56.215118 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:56.215545 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:56.215549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:56.215556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:56.216070 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:56.216381 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:56.216867 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 22:54:56.217335 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":385,"threads_started":1}
I20260807 22:54:56.225150 29578 raft_consensus.cc:348] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } }
I20260807 22:54:56.225545 29578 raft_consensus.cc:374] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:56.225656 29578 raft_consensus.cc:729] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c237efd2e4ce438ca6f3b13182e5bce8, State: Initialized, Role: FOLLOWER
I20260807 22:54:56.226209 29578 consensus_queue.cc:260] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } }
I20260807 22:54:56.226466 29578 raft_consensus.cc:388] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:56.226619 29578 raft_consensus.cc:482] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:56.226799 29578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:56.229271 29578 raft_consensus.cc:504] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } }
I20260807 22:54:56.229773 29578 leader_election.cc:302] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c237efd2e4ce438ca6f3b13182e5bce8; no voters:
I20260807 22:54:56.230429 29578 leader_election.cc:288] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:54:56.230600 29583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:56.231882 29583 raft_consensus.cc:686] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [term 1 LEADER]: Becoming Leader. State: Replica: c237efd2e4ce438ca6f3b13182e5bce8, State: Running, Role: LEADER
I20260807 22:54:56.232409 29583 consensus_queue.cc:237] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } }
I20260807 22:54:56.232859 29578 sys_catalog.cc:564] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:56.235344 29585 sys_catalog.cc:455] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } } }
I20260807 22:54:56.235796 29585 sys_catalog.cc:458] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:56.235522 29586 sys_catalog.cc:455] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c237efd2e4ce438ca6f3b13182e5bce8. Latest consensus state: current_term: 1 leader_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c237efd2e4ce438ca6f3b13182e5bce8" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 32995 } } }
I20260807 22:54:56.235977 29586 sys_catalog.cc:458] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:56.239216 29593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:56.244562 29593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:56.253785 29593 catalog_manager.cc:1347] Generated new cluster ID: 158c1f73337c4ab3a9dfcaecb24a95ac
I20260807 22:54:56.253928 29593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:56.269760 29593 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:54:56.271121 29593 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:56.279187 29593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: Generated new TSK 0
I20260807 22:54:56.280019 29593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:54:56.367405 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:56.720432 29608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:56.720710 29608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:56.720862 29608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:56.728220 29608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:56.728331 29608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:56.728523 29608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:56.739413 29608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:56.740772 29608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:56.742148 29608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:56.756081 29615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:56.756201 29614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:56.756593 29618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:56.762259 29608 server_base.cc:1034] running on GCE node
I20260807 22:54:56.762905 29608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:56.764070 29608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:56.765278 29608 hybrid_clock.cc:648] HybridClock initialized: now 1786143296765228 us; error 37 us; skew 500 ppm
I20260807 22:54:56.765630 29608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:56 dist-test-slave-5k54 krb5kdc[29492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:56.774552 29608 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:54:56.776309 29608 webserver.cc:460] Webserver started at http://127.9.194.193:33059/ using document root <none> and password file <none>
I20260807 22:54:56.777091 29608 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:56.777199 29608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:56.777568 29608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:56.780925 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "b75f51afeb6f4528dd1e9f95de31877b"
server_key_iv: "b17678d2a91e2082edaea7aaa2c51393"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.781750 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "b75f51afeb6f4528dd1e9f95de31877b"
server_key_iv: "b17678d2a91e2082edaea7aaa2c51393"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.787140 29608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:54:56.790818 29625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:56.792224 29608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:56.792438 29608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce"
format_stamp: "Formatted at 2026-08-07 22:54:56 on dist-test-slave-5k54"
server_key: "b75f51afeb6f4528dd1e9f95de31877b"
server_key_iv: "b17678d2a91e2082edaea7aaa2c51393"
server_key_version: "encryptionkey@0"
I20260807 22:54:56.792660 29608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:56.840785 29608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:56.845507 29608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:56.845810 29608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:56.846832 29608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:56.848265 29608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:56.848364 29608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:56.848517 29608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:56.848589 29608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:56.884263 29608 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:42735
I20260807 22:54:56.884301 29748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:42735 every 8 connection(s)
I20260807 22:54:56.885661 29608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:54:56.893075 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29608
I20260807 22:54:56.893265 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:54:56.893712 9995 external_mini_cluster.cc:1428] Setting key 9d757b85c1456f02f734b5bff41bad51
Aug 07 22:54:56 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:56.908834 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:56.888332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:58465 (local address 127.9.194.254:32995)
0807 22:54:56.888793 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:56.888798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:56.889753 (+ 955us) server_negotiation.cc:407] Connection header received
0807 22:54:56.891678 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:56.891684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:56.891777 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:56.891902 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:56.893515 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.894373 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.896503 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.896983 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.897174 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:56.901280 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:56.901290 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:56.901298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:56.901345 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:56.905209 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:56.905997 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:56.906005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:56.906011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:56.906164 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:56.906605 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:56.906612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:56.906618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:56.907022 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:56.907175 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:56.908066 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 22:54:56.908339 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":251,"threads_started":1}
I20260807 22:54:56.909994 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:56.888846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:32995 (local address 127.9.194.193:58465)
0807 22:54:56.889565 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:56.889621 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:56.891349 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:56.892145 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:56.892153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:56.892687 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:56.893325 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:56.893353 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.894559 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.894564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:56.896358 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:56.896377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:56.897224 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:56.897235 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:56.897581 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:56.898332 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:56.898373 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:56.901057 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:56.905378 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:56.905385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:56.905399 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:56.905852 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:56.906275 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:56.906278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:56.906280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:56.906485 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:56.907148 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:56.907158 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:56.907623 (+ 465us) client_negotiation.cc:769] Sending connection context
0807 22:54:56.907913 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 22:54:56.908486 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":272,"threads_started":1}
I20260807 22:54:56.911741 29749 heartbeater.cc:347] Connected to a master server at 127.9.194.254:32995
I20260807 22:54:56.912070 29749 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:56.913007 29749 heartbeater.cc:511] Master 127.9.194.254:32995 requested a full tablet report, sending...
I20260807 22:54:56.915166 29542 ts_manager.cc:194] Registered new tserver with Master: e3ee62119e764e3e8e1b5ce846a9f2ce (127.9.194.193:42735)
I20260807 22:54:56.916667 29542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:58465
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:54:56.962731 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:57.318406 29757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:57.318676 29757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:57.318858 29757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:57.326167 29757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:57.326269 29757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:57.326459 29757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:54:57.336449 29757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:57.337725 29757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:57.339169 29757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:57.352766 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:57.352860 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:57.352818 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:57.353726 29757 server_base.cc:1034] running on GCE node
I20260807 22:54:57.354291 29757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:57.355187 29757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:57.356384 29757 hybrid_clock.cc:648] HybridClock initialized: now 1786143297356314 us; error 63 us; skew 500 ppm
I20260807 22:54:57.356710 29757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:57 dist-test-slave-5k54 krb5kdc[29492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:57.360014 29757 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:54:57.361483 29757 webserver.cc:460] Webserver started at http://127.9.194.194:43355/ using document root <none> and password file <none>
I20260807 22:54:57.362172 29757 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:57.362313 29757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:57.362620 29757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:57.365530 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "2e380ddb008e4b7fbe87d4e247379922"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "639183f0294e8019fc5a66c0afecdcf1"
server_key_iv: "fc6cc08fdbfd5e50d501f8b1fcef3100"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.366297 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "2e380ddb008e4b7fbe87d4e247379922"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "639183f0294e8019fc5a66c0afecdcf1"
server_key_iv: "fc6cc08fdbfd5e50d501f8b1fcef3100"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.371553 29757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260807 22:54:57.374972 29774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:57.376587 29757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:57.376751 29757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "2e380ddb008e4b7fbe87d4e247379922"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "639183f0294e8019fc5a66c0afecdcf1"
server_key_iv: "fc6cc08fdbfd5e50d501f8b1fcef3100"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.377079 29757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:57.421005 29757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:57.425235 29757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:57.425539 29757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:57.426548 29757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:57.427960 29757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:57.428033 29757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:57.428186 29757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:57.428242 29757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:57.462479 29757 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:34079
I20260807 22:54:57.462531 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:34079 every 8 connection(s)
I20260807 22:54:57.463806 29757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:54:57.464128 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29757
I20260807 22:54:57.464324 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:54:57.464962 9995 external_mini_cluster.cc:1428] Setting key 49bba9da0364aa33d6704cea85c6f6db
Aug 07 22:54:57 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:57.484812 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:57.466609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:41157 (local address 127.9.194.254:32995)
0807 22:54:57.467111 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:57.467117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:57.468207 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 22:54:57.470126 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:57.470131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:57.470221 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:57.470334 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:57.471939 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:57.472705 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:57.474229 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:57.474664 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:57.474796 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:57.478484 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:57.478488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:57.478492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:57.478529 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:57.481711 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:57.482459 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:57.482465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:57.482468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:57.482603 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:57.483048 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:57.483053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:57.483060 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:57.483322 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:57.483517 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:57.484184 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 22:54:57.484405 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":238,"threads_started":1}
I20260807 22:54:57.485750 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:57.467269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:32995 (local address 127.9.194.194:41157)
0807 22:54:57.468058 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:57.468098 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:57.469846 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:57.470591 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:57.470600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:57.471121 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:57.471766 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:57.471785 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:57.472867 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:57.472872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:57.474077 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:57.474089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:57.474888 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:57.474897 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:57.475134 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:57.475841 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:57.475869 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:57.478241 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:57.481875 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:57.481884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:57.481899 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:57.482322 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:57.482721 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:57.482725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:57.482729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:57.482930 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:57.483477 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:57.483484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:57.483823 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 22:54:57.484083 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 22:54:57.484610 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":307,"threads_started":1}
I20260807 22:54:57.487358 29892 heartbeater.cc:347] Connected to a master server at 127.9.194.254:32995
I20260807 22:54:57.487746 29892 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:57.488631 29892 heartbeater.cc:511] Master 127.9.194.254:32995 requested a full tablet report, sending...
I20260807 22:54:57.490319 29542 ts_manager.cc:194] Registered new tserver with Master: 2e380ddb008e4b7fbe87d4e247379922 (127.9.194.194:34079)
I20260807 22:54:57.491259 29542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:41157
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:54:57.535204 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:54:57.889953 29899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:57.890213 29899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:57.890383 29899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:57.897928 29899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:57.898032 29899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:57.898222 29899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:54:57.907995 29899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:32995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:57.909291 29899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:57.910590 29899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:54:57.920395 29749 heartbeater.cc:503] Master 127.9.194.254:32995 was elected leader, sending a full tablet report...
W20260807 22:54:57.924567 29904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:57.924804 29906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:57.924768 29908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:57.929030 29899 server_base.cc:1034] running on GCE node
I20260807 22:54:57.929636 29899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:57.930565 29899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:57.931780 29899 hybrid_clock.cc:648] HybridClock initialized: now 1786143297931736 us; error 36 us; skew 500 ppm
I20260807 22:54:57.932120 29899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:54:57 dist-test-slave-5k54 krb5kdc[29492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:54:57.935317 29899 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:54:57.936757 29899 webserver.cc:460] Webserver started at http://127.9.194.195:43093/ using document root <none> and password file <none>
I20260807 22:54:57.937485 29899 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:57.937587 29899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:57.937848 29899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:57.940805 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "54f22716e5a54f03ab0f85de9b419fd3"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "c3991390f639e57b389030312ece7b62"
server_key_iv: "22b574b514baaac5de669c20b9d0b163"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.941541 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "54f22716e5a54f03ab0f85de9b419fd3"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "c3991390f639e57b389030312ece7b62"
server_key_iv: "22b574b514baaac5de669c20b9d0b163"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.946570 29899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:54:57.949925 29915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:57.951174 29899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:54:57.951310 29899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "54f22716e5a54f03ab0f85de9b419fd3"
format_stamp: "Formatted at 2026-08-07 22:54:57 on dist-test-slave-5k54"
server_key: "c3991390f639e57b389030312ece7b62"
server_key_iv: "22b574b514baaac5de669c20b9d0b163"
server_key_version: "encryptionkey@0"
I20260807 22:54:57.951538 29899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:57.982417 29899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:57.986519 29899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:57.986804 29899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:57.987788 29899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:57.989085 29899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:57.989161 29899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:57.989343 29899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:57.989409 29899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:58.024056 29899 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:43661
I20260807 22:54:58.024098 30031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:43661 every 8 connection(s)
I20260807 22:54:58.025512 29899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:54:58.026268 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 29899
I20260807 22:54:58.026460 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:54:58.027102 9995 external_mini_cluster.cc:1428] Setting key e9b339badc13cf5112ba1a1b04e45148
Aug 07 22:54:58 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:54:58.047370 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.028907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:43215 (local address 127.9.194.254:32995)
0807 22:54:58.029392 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.029397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.030641 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 22:54:58.032540 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.032546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.032641 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.032773 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:58.034652 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.035536 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.037087 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.037498 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.037673 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.041271 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:58.041278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:58.041284 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:58.041329 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:58.044578 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.045353 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.045357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.045360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.045483 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.045878 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.045888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.045890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.046142 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:58.046258 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.046906 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.047065 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":235,"threads_started":1}
I20260807 22:54:58.048285 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.029637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:32995 (local address 127.9.194.195:43215)
0807 22:54:58.030490 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.030529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.032232 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.033037 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.033046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.033946 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:54:58.034499 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.034516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.035685 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.035689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.036960 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.036972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.037769 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.037774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.038002 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.038634 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:54:58.038655 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:54:58.041007 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:54:58.044746 (+ 3739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:58.044753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:58.044764 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:58.045236 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:58.045584 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:54:58.045587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:54:58.045589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:54:58.045762 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:54:58.046255 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:54:58.046260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:54:58.046568 (+ 308us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.046802 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.047181 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":348,"threads_started":1}
I20260807 22:54:58.049608 30032 heartbeater.cc:347] Connected to a master server at 127.9.194.254:32995
I20260807 22:54:58.049923 30032 heartbeater.cc:464] Registering TS with master...
I20260807 22:54:58.050786 30032 heartbeater.cc:511] Master 127.9.194.254:32995 requested a full tablet report, sending...
I20260807 22:54:58.052333 29542 ts_manager.cc:194] Registered new tserver with Master: 54f22716e5a54f03ab0f85de9b419fd3 (127.9.194.195:43661)
I20260807 22:54:58.053089 29542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:43215
I20260807 22:54:58.055317 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:54:58.073304 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.059799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.9.194.254:32995)
0807 22:54:58.060022 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.060030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.060407 (+ 377us) server_negotiation.cc:407] Connection header received
0807 22:54:58.060598 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.060603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.060675 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.060775 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:58.062153 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.062866 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.064214 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.064619 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.064754 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.067509 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:58.067516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:58.067520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:58.067562 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:58.069935 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.070859 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.070866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.070872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.071180 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.071789 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.071794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.071798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.072101 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:58.072391 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.072790 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.073027 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:54:58.077396 29542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:54:58.084653 29542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:54:58.114598 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.098726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.9.194.193:42735)
0807 22:54:58.103569 (+ 4843us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.103581 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.103620 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:54:58.103745 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.103756 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.104104 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.104332 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:58.107739 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.109396 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.112122 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.113088 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.113246 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.113502 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.114054 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.114225 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4578,"thread_start_us":4348,"threads_started":1}
I20260807 22:54:58.114598 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.098020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36874)
0807 22:54:58.102616 (+ 4596us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.102640 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.102784 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.104685 (+ 1901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.104697 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.104740 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:58.107500 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.107524 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.109552 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.109557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.111924 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.111936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.113680 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.113690 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.113814 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.113851 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.114001 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.114212 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3002,"thread_start_us":8453,"threads_started":1}
I20260807 22:54:58.116775 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.097600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43661 (local address 127.0.0.1:58184)
0807 22:54:58.100323 (+ 2723us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.100368 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.100532 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.108784 (+ 8252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.108789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.108814 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:58.109228 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.109239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.111504 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.111509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.114108 (+ 2599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.114118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.115915 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.115921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.116087 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.116120 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.116240 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.116377 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2479,"mutex_wait_us":195,"thread_start_us":2343,"threads_started":1}
I20260807 22:54:58.116773 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.097979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.9.194.195:43661)
0807 22:54:58.108252 (+ 10273us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.108260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.108291 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:54:58.108381 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.108389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.108601 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.108807 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:58.109637 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.111286 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.114290 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.115710 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.115859 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.116105 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.116285 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.116432 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10060,"thread_start_us":9873,"threads_started":1}
I20260807 22:54:58.117965 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.099976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:34079 (local address 127.0.0.1:58450)
0807 22:54:58.106812 (+ 6836us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.106847 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.106986 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.109073 (+ 2087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.109077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.109111 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:58.109531 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.109548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.111390 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.111399 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.114097 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.114112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.117129 (+ 3017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.117138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.117316 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.117346 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.117446 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.117563 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6684,"mutex_wait_us":111,"thread_start_us":867,"threads_started":1}
I20260807 22:54:58.118012 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.100800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.9.194.194:34079)
0807 22:54:58.108486 (+ 7686us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.108494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.108521 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:54:58.108619 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.108626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.108857 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.109760 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:58.110048 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.111239 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.114289 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.116620 (+ 2331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.116764 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.116959 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.117507 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.117654 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7530,"thread_start_us":7400,"threads_started":1}
I20260807 22:54:58.119254 29677 tablet_service.cc:1467] Processing CreateTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 (DEFAULT_TABLE table=test-table [id=ed8a5efc5a0b4b2d9f9b2e4c0be212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:58.120893 29677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e95d59d78e4d4b928d6293c18d8856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:58.121945 29964 tablet_service.cc:1467] Processing CreateTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 (DEFAULT_TABLE table=test-table [id=ed8a5efc5a0b4b2d9f9b2e4c0be212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:58.122051 29825 tablet_service.cc:1467] Processing CreateTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 (DEFAULT_TABLE table=test-table [id=ed8a5efc5a0b4b2d9f9b2e4c0be212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:54:58.123204 29964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e95d59d78e4d4b928d6293c18d8856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:58.123409 29825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e95d59d78e4d4b928d6293c18d8856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:58.134758 30052 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Bootstrap starting.
I20260807 22:54:58.136590 30054 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Bootstrap starting.
I20260807 22:54:58.140367 30052 tablet_bootstrap.cc:654] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:58.140431 30054 tablet_bootstrap.cc:654] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:58.141690 30052 log.cc:824] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:58.141690 30054 log.cc:824] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:58.142670 30055 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Bootstrap starting.
I20260807 22:54:58.146215 30055 tablet_bootstrap.cc:654] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:58.146245 30054 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: No bootstrap required, opened a new log
I20260807 22:54:58.146531 30054 ts_tablet_manager.cc:1397] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 22:54:58.147161 30052 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: No bootstrap required, opened a new log
I20260807 22:54:58.147454 30052 ts_tablet_manager.cc:1397] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260807 22:54:58.147415 30055 log.cc:824] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:58.150296 30055 tablet_bootstrap.cc:492] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: No bootstrap required, opened a new log
I20260807 22:54:58.150543 30055 ts_tablet_manager.cc:1397] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:54:58.157505 30054 raft_consensus.cc:348] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.157958 30054 raft_consensus.cc:374] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:58.158061 30054 raft_consensus.cc:729] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ee62119e764e3e8e1b5ce846a9f2ce, State: Initialized, Role: FOLLOWER
I20260807 22:54:58.158568 30055 raft_consensus.cc:348] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.158702 30054 consensus_queue.cc:260] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.158910 30055 raft_consensus.cc:374] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:58.159006 30055 raft_consensus.cc:729] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e380ddb008e4b7fbe87d4e247379922, State: Initialized, Role: FOLLOWER
I20260807 22:54:58.158828 30052 raft_consensus.cc:348] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.159243 30052 raft_consensus.cc:374] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:58.159444 30052 raft_consensus.cc:729] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f22716e5a54f03ab0f85de9b419fd3, State: Initialized, Role: FOLLOWER
I20260807 22:54:58.159663 30055 consensus_queue.cc:260] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.160735 29892 heartbeater.cc:503] Master 127.9.194.254:32995 was elected leader, sending a full tablet report...
I20260807 22:54:58.161682 30055 ts_tablet_manager.cc:1428] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260807 22:54:58.162276 30054 ts_tablet_manager.cc:1428] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260807 22:54:58.162343 30052 consensus_queue.cc:260] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.163478 30032 heartbeater.cc:503] Master 127.9.194.254:32995 was elected leader, sending a full tablet report...
I20260807 22:54:58.164017 30052 ts_tablet_manager.cc:1428] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Time spent starting tablet: real 0.016s user 0.012s sys 0.001s
W20260807 22:54:58.216255 29894 tablet.cc:2369] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:54:58.267865 30060 raft_consensus.cc:482] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:54:58.268232 30060 raft_consensus.cc:504] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.269915 30060 leader_election.cc:288] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54f22716e5a54f03ab0f85de9b419fd3 (127.9.194.195:43661), e3ee62119e764e3e8e1b5ce846a9f2ce (127.9.194.193:42735)
I20260807 22:54:58.277002 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.270398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:34113 (local address 127.9.194.195:43661)
0807 22:54:58.270618 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.270625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.271507 (+ 882us) server_negotiation.cc:407] Connection header received
0807 22:54:58.271757 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.271761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.271856 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.271953 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:58.272587 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.273290 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.275189 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.276029 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.276189 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.276277 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.276609 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.276738 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:54:58.277014 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.270501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:37287 (local address 127.9.194.193:42735)
0807 22:54:58.270694 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.270702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.271481 (+ 779us) server_negotiation.cc:407] Connection header received
0807 22:54:58.271649 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.271654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.271719 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.271802 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:54:58.272512 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.273279 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.275227 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.276040 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.276228 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.276300 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.276645 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.276763 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:54:58.277173 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.270360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43661 (local address 127.9.194.194:34113)
0807 22:54:58.271383 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.271397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.271601 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.272001 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.272005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.272041 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:58.272406 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.272414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.273462 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.273466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.275013 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.275028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.276260 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.276270 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.276431 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.276474 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.276547 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.276695 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":4,"thread_start_us":443,"threads_started":1}
I20260807 22:54:58.277302 30063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.270295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.9.194.194:37287)
0807 22:54:58.271278 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.271300 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:54:58.271501 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:54:58.271836 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:54:58.271841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:54:58.271864 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:54:58.272351 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.272364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.273411 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.273418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.275043 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:54:58.275055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.276287 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.276295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:54:58.276443 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.276490 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:54:58.276598 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:54:58.276725 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":112,"spinlock_wait_cycles":25856,"thread_start_us":667,"threads_started":1}
W20260807 22:54:58.277899 30033 tablet.cc:2369] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:54:58.278375 29985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e95d59d78e4d4b928d6293c18d8856" candidate_uuid: "2e380ddb008e4b7fbe87d4e247379922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54f22716e5a54f03ab0f85de9b419fd3" is_pre_election: true
I20260807 22:54:58.278410 29700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e95d59d78e4d4b928d6293c18d8856" candidate_uuid: "2e380ddb008e4b7fbe87d4e247379922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" is_pre_election: true
I20260807 22:54:58.278899 29985 raft_consensus.cc:2454] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e380ddb008e4b7fbe87d4e247379922 in term 0.
I20260807 22:54:58.278972 29700 raft_consensus.cc:2454] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e380ddb008e4b7fbe87d4e247379922 in term 0.
I20260807 22:54:58.279628 29777 leader_election.cc:302] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e380ddb008e4b7fbe87d4e247379922, 54f22716e5a54f03ab0f85de9b419fd3; no voters:
I20260807 22:54:58.280010 30060 raft_consensus.cc:2781] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:54:58.280167 30060 raft_consensus.cc:482] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:54:58.280277 30060 raft_consensus.cc:3037] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:58.282541 30060 raft_consensus.cc:504] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.283165 30060 leader_election.cc:288] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [CANDIDATE]: Term 1 election: Requested vote from peers 54f22716e5a54f03ab0f85de9b419fd3 (127.9.194.195:43661), e3ee62119e764e3e8e1b5ce846a9f2ce (127.9.194.193:42735)
I20260807 22:54:58.283591 29985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e95d59d78e4d4b928d6293c18d8856" candidate_uuid: "2e380ddb008e4b7fbe87d4e247379922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54f22716e5a54f03ab0f85de9b419fd3"
I20260807 22:54:58.283741 29700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e95d59d78e4d4b928d6293c18d8856" candidate_uuid: "2e380ddb008e4b7fbe87d4e247379922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce"
I20260807 22:54:58.283787 29985 raft_consensus.cc:3037] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:58.283960 29700 raft_consensus.cc:3037] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:58.286018 29985 raft_consensus.cc:2454] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e380ddb008e4b7fbe87d4e247379922 in term 1.
I20260807 22:54:58.286130 29700 raft_consensus.cc:2454] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e380ddb008e4b7fbe87d4e247379922 in term 1.
I20260807 22:54:58.286409 29777 leader_election.cc:302] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e380ddb008e4b7fbe87d4e247379922, 54f22716e5a54f03ab0f85de9b419fd3; no voters:
I20260807 22:54:58.286701 30060 raft_consensus.cc:2781] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:58.287173 30060 raft_consensus.cc:686] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 1 LEADER]: Becoming Leader. State: Replica: 2e380ddb008e4b7fbe87d4e247379922, State: Running, Role: LEADER
I20260807 22:54:58.287704 30060 consensus_queue.cc:237] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } }
I20260807 22:54:58.292989 29540 catalog_manager.cc:5495] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e380ddb008e4b7fbe87d4e247379922 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "2e380ddb008e4b7fbe87d4e247379922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e380ddb008e4b7fbe87d4e247379922" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 34079 } health_report { overall_health: HEALTHY } } }
I20260807 22:54:58.347806 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.342413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.9.194.194:34079)
0807 22:54:58.342594 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.342599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.342768 (+ 169us) server_negotiation.cc:407] Connection header received
0807 22:54:58.342948 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.342952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.343032 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.343159 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:54:58.343622 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.344341 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.345732 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.346123 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.346214 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.346737 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:54:58.347251 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:54:58.347376 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.347565 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.347603 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:54:58.356252 29700 raft_consensus.cc:1261] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 1 FOLLOWER]: Refusing update from remote peer 2e380ddb008e4b7fbe87d4e247379922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:58.356274 29985 raft_consensus.cc:1261] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 1 FOLLOWER]: Refusing update from remote peer 2e380ddb008e4b7fbe87d4e247379922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:54:58.357465 30064 consensus_queue.cc:1035] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54f22716e5a54f03ab0f85de9b419fd3" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:58.358237 30060 consensus_queue.cc:1035] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3ee62119e764e3e8e1b5ce846a9f2ce" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 42735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:54:58.385991 29540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:58.386281 29540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:54:58.389890 29540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c237efd2e4ce438ca6f3b13182e5bce8: Sending DeleteTablet for 3 replicas of tablet 31e95d59d78e4d4b928d6293c18d8856
I20260807 22:54:58.391014 29964 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:58 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:58184
I20260807 22:54:58.391057 29677 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:58 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:36874
I20260807 22:54:58.391458 29825 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e95d59d78e4d4b928d6293c18d8856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:54:58 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:58450
I20260807 22:54:58.392452 30080 tablet_replica.cc:330] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: stopping tablet replica
I20260807 22:54:58.393047 30080 raft_consensus.cc:2229] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:54:58.393272 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29608
I20260807 22:54:58.393786 30080 raft_consensus.cc:2258] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:58.396309 30080 ts_tablet_manager.cc:1905] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:58.405622 30080 ts_tablet_manager.cc:1918] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:58.405884 30080 log.cc:1196] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/31e95d59d78e4d4b928d6293c18d8856
I20260807 22:54:58.406541 30080 ts_tablet_manager.cc:1939] T 31e95d59d78e4d4b928d6293c18d8856 P 2e380ddb008e4b7fbe87d4e247379922: Deleting consensus metadata
I20260807 22:54:58.408540 29525 catalog_manager.cc:4841] TS 2e380ddb008e4b7fbe87d4e247379922 (127.9.194.194:34079): tablet 31e95d59d78e4d4b928d6293c18d8856 (table test-table [id=ed8a5efc5a0b4b2d9f9b2e4c0be212ad]) successfully deleted
Aug 07 22:54:58 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:54:58.409346 30079 tablet_replica.cc:330] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: stopping tablet replica
I20260807 22:54:58.409729 30078 tablet_replica.cc:330] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: stopping tablet replica
I20260807 22:54:58.410125 30078 raft_consensus.cc:2229] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:58.410610 30078 raft_consensus.cc:2258] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:58.411281 30079 raft_consensus.cc:2229] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:54:58.411849 30079 raft_consensus.cc:2258] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:54:58.413539 30078 ts_tablet_manager.cc:1905] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:58.413863 30079 ts_tablet_manager.cc:1905] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:54:58.418442 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.394374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.9.194.193:42735)
0807 22:54:58.394846 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.394852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.394998 (+ 146us) server_negotiation.cc:407] Connection header received
0807 22:54:58.395311 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.395319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.395458 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.395552 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:58.398067 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.399664 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.402293 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.403267 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.403392 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.410714 (+ 7322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:58.410729 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:58.410746 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:58.410791 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:58.414803 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.415722 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.415731 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.415734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.415901 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.416371 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.416378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.416381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.416812 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:58.417071 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.417727 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.417972 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260807 22:54:58.419879 29744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:58.421008 30079 ts_tablet_manager.cc:1918] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:58.421231 30079 log.cc:1196] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/31e95d59d78e4d4b928d6293c18d8856
I20260807 22:54:58.421820 30079 ts_tablet_manager.cc:1939] T 31e95d59d78e4d4b928d6293c18d8856 P 54f22716e5a54f03ab0f85de9b419fd3: Deleting consensus metadata
I20260807 22:54:58.424098 29526 catalog_manager.cc:4841] TS 54f22716e5a54f03ab0f85de9b419fd3 (127.9.194.195:43661): tablet 31e95d59d78e4d4b928d6293c18d8856 (table test-table [id=ed8a5efc5a0b4b2d9f9b2e4c0be212ad]) successfully deleted
I20260807 22:54:58.519358 30078 ts_tablet_manager.cc:1918] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:54:58.519656 30078 log.cc:1196] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/31e95d59d78e4d4b928d6293c18d8856
I20260807 22:54:58.520117 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29608
I20260807 22:54:58.520239 30078 ts_tablet_manager.cc:1939] T 31e95d59d78e4d4b928d6293c18d8856 P e3ee62119e764e3e8e1b5ce846a9f2ce: Deleting consensus metadata
I20260807 22:54:58.545646 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29757
W20260807 22:54:58.545717 29527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:42735 (error 108)
I20260807 22:54:58.549439 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.548919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36882)
0807 22:54:58.549091 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.549241 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 22:54:58.550439 29527 catalog_manager.cc:4584] TS e3ee62119e764e3e8e1b5ce846a9f2ce (127.9.194.193:42735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31e95d59d78e4d4b928d6293c18d8856: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Aug 07 22:54:58 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:54:58.564241 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.546879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.9.194.194:34079)
0807 22:54:58.547121 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.547130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.547154 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:54:58.547261 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.547269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.547409 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.547587 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:58.549266 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.550227 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.552133 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.552582 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.552680 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.558139 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:58.558161 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:58.558177 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:58.558214 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:58.561259 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.562001 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.562010 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.562014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.562168 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.562552 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.562558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.562562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.562975 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:58.563316 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.563657 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.563891 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:54:58.564954 29887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:58.604339 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.603855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36890)
0807 22:54:58.604072 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.604167 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:54:58.605422 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.605070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36894)
0807 22:54:58.605199 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.605270 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:54:58.666117 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.665631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36898)
0807 22:54:58.665849 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.665948 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 22:54:58.667142 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.666834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36914)
0807 22:54:58.666962 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.667026 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:54:58.675755 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29757
I20260807 22:54:58.696194 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29899
Aug 07 22:54:58 dist-test-slave-5k54 krb5kdc[29492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:54:58.711539 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.697024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.9.194.195:43661)
0807 22:54:58.697244 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:58.697251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:58.697271 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:54:58.697449 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:58.697456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:58.697531 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:58.697638 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:58.698995 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.699841 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.701137 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:58.701547 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:58.701678 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:58.706183 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:58.706197 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:58.706212 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:54:58.706246 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:58.709035 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.709704 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.709712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.709714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.709857 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:54:58.710199 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:54:58.710202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:54:58.710204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:54:58.710559 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:58.710773 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:58.711000 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 22:54:58.711168 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:54:58.712185 30027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:58.747550 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.747030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36922)
0807 22:54:58.747226 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.747319 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 22:54:58.748629 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.748239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36934)
0807 22:54:58.748446 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.748516 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:54:58.801440 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29899
I20260807 22:54:58.820104 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 29506
I20260807 22:54:58.821261 29573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:54:58.935865 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 29506
I20260807 22:54:58.935896 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:58.935294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:42735 (local address 127.0.0.1:36946)
0807 22:54:58.935556 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:54:58.935665 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:42735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
2026-08-07T22:54:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4659 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T22:54:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:54:58Z Disabled control of system clock
I20260807 22:54:59.026885 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41745
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:41745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:54:59.363453 30091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:59.363749 30091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:59.363837 30091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:59.370929 30091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:54:59.371040 30091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:59.371079 30091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:59.371138 30091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:54:59.371183 30091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:54:59.380155 30091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:41745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:59.381403 30091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:59.383109 30091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:59.395028 30098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:59.395174 30097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:59.395141 30100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:59.396476 30091 server_base.cc:1034] running on GCE node
I20260807 22:54:59.397256 30091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:59.398612 30091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:59.399844 30091 hybrid_clock.cc:648] HybridClock initialized: now 1786143299399806 us; error 49 us; skew 500 ppm
I20260807 22:54:59.400223 30091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:59.402519 30091 webserver.cc:460] Webserver started at http://127.9.194.254:32807/ using document root <none> and password file <none>
I20260807 22:54:59.403205 30091 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:59.403340 30091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:59.403692 30091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:59.406565 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "c003564bd2804514bf9a6f43c9074332"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "6bfa107df7400cb5be5acc137feaad26"
server_key_iv: "5e8079f95fb1a3732610071803a60b45"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.407353 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "c003564bd2804514bf9a6f43c9074332"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "6bfa107df7400cb5be5acc137feaad26"
server_key_iv: "5e8079f95fb1a3732610071803a60b45"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.412312 30091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:54:59.415591 30106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:59.416774 30091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:54:59.416987 30091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "c003564bd2804514bf9a6f43c9074332"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "6bfa107df7400cb5be5acc137feaad26"
server_key_iv: "5e8079f95fb1a3732610071803a60b45"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.417200 30091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:59.439704 30091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:59.440567 30091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:59.440871 30091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:59.460105 30091 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:41745
I20260807 22:54:59.460131 30163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:41745 every 8 connection(s)
I20260807 22:54:59.461462 30091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:54:59.463356 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30091
I20260807 22:54:59.463593 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:54:59.464107 9995 external_mini_cluster.cc:1428] Setting key 41d03a57dd6a269f9470e63955c0870c
I20260807 22:54:59.465597 30164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:54:59.477149 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:54:59.465996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.9.194.254:41745)
0807 22:54:59.466962 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 22:54:59.466973 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:54:59.467013 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:54:59.468661 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:54:59.468681 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:54:59.469125 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:54:59.469542 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:54:59.470096 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:59.471473 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:59.472877 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:54:59.473521 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:54:59.473714 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:54:59.474908 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:54:59.474916 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:54:59.474929 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:54:59.475548 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:54:59.475609 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:54:59.475635 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:54:59.475762 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:54:59.475981 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 22:54:59.476257 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":357,"threads_started":1}
I20260807 22:54:59.477252 30164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: Bootstrap starting.
I20260807 22:54:59.480768 30164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: Neither blocks nor log segments found. Creating new log.
I20260807 22:54:59.481890 30164 log.cc:824] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: Log is configured to *not* fsync() on all Append() calls
I20260807 22:54:59.484724 30164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: No bootstrap required, opened a new log
I20260807 22:54:59.491990 30164 raft_consensus.cc:348] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } }
I20260807 22:54:59.492331 30164 raft_consensus.cc:374] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:54:59.492460 30164 raft_consensus.cc:729] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c003564bd2804514bf9a6f43c9074332, State: Initialized, Role: FOLLOWER
I20260807 22:54:59.493007 30164 consensus_queue.cc:260] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } }
I20260807 22:54:59.493204 30164 raft_consensus.cc:388] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:54:59.493356 30164 raft_consensus.cc:482] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:54:59.493526 30164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:54:59.495877 30164 raft_consensus.cc:504] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } }
I20260807 22:54:59.496359 30164 leader_election.cc:302] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c003564bd2804514bf9a6f43c9074332; no voters:
I20260807 22:54:59.496937 30164 leader_election.cc:288] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:54:59.497099 30169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:54:59.497823 30169 raft_consensus.cc:686] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [term 1 LEADER]: Becoming Leader. State: Replica: c003564bd2804514bf9a6f43c9074332, State: Running, Role: LEADER
I20260807 22:54:59.498409 30169 consensus_queue.cc:237] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } }
I20260807 22:54:59.499011 30164 sys_catalog.cc:564] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:54:59.501480 30171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c003564bd2804514bf9a6f43c9074332. Latest consensus state: current_term: 1 leader_uuid: "c003564bd2804514bf9a6f43c9074332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } } }
I20260807 22:54:59.501955 30171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:59.502086 30170 sys_catalog.cc:455] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c003564bd2804514bf9a6f43c9074332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c003564bd2804514bf9a6f43c9074332" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 41745 } } }
I20260807 22:54:59.502461 30170 sys_catalog.cc:458] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332 [sys.catalog]: This master's current role is: LEADER
I20260807 22:54:59.512672 30177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:54:59.517920 30177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:54:59.527673 30177 catalog_manager.cc:1347] Generated new cluster ID: 4bbd47d243d5407db45e9c870aaf070f
I20260807 22:54:59.527827 30177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:54:59.549443 30177 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:54:59.550710 30177 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:54:59.562659 30177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: Generated new TSK 0
I20260807 22:54:59.563540 30177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:54:59.575541 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--builtin_ntp_servers=127.9.194.212:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:54:59.917649 30189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:54:59.917975 30189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:54:59.918133 30189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:54:59.925570 30189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:54:59.925668 30189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:54:59.925773 30189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:54:59.934688 30189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:54:59.935988 30189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:54:59.937697 30189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:54:59.950631 30194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:59.950601 30197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:54:59.950672 30195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:54:59.951646 30189 server_base.cc:1034] running on GCE node
I20260807 22:54:59.952239 30189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:54:59.953234 30189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:54:59.954487 30189 hybrid_clock.cc:648] HybridClock initialized: now 1786143299954504 us; error 93 us; skew 500 ppm
I20260807 22:54:59.954882 30189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:54:59.957283 30189 webserver.cc:460] Webserver started at http://127.9.194.193:34627/ using document root <none> and password file <none>
I20260807 22:54:59.957975 30189 fs_manager.cc:359] Metadata directory not provided
I20260807 22:54:59.958109 30189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:54:59.958441 30189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:54:59.961334 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "864fd3ecdc8d485b8867449e58a48791"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "2e1f17e400e45896958639f6e81e51b0"
server_key_iv: "ea88b08b91b16854862d240e9c42cfb7"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.962159 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "864fd3ecdc8d485b8867449e58a48791"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "2e1f17e400e45896958639f6e81e51b0"
server_key_iv: "ea88b08b91b16854862d240e9c42cfb7"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.967361 30189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:54:59.970786 30203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:59.972178 30189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:54:59.972354 30189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "864fd3ecdc8d485b8867449e58a48791"
format_stamp: "Formatted at 2026-08-07 22:54:59 on dist-test-slave-5k54"
server_key: "2e1f17e400e45896958639f6e81e51b0"
server_key_iv: "ea88b08b91b16854862d240e9c42cfb7"
server_key_version: "encryptionkey@0"
I20260807 22:54:59.972579 30189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:54:59.992973 30189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:54:59.993808 30189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:54:59.994128 30189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:54:59.995105 30189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:54:59.996801 30189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:54:59.996922 30189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:54:59.997051 30189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:54:59.997148 30189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.033377 30189 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:34507
I20260807 22:55:00.033424 30326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:34507 every 8 connection(s)
I20260807 22:55:00.034749 30189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:55:00.038637 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30189
I20260807 22:55:00.038843 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:55:00.039309 9995 external_mini_cluster.cc:1428] Setting key 04353dce2ace72bcbfac13dcc2347b9a
I20260807 22:55:00.042475 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--builtin_ntp_servers=127.9.194.212:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:55:00.049330 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:00.037458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:35255 (local address 127.9.194.254:41745)
0807 22:55:00.037921 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:00.037926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:00.039330 (+ 1404us) server_negotiation.cc:407] Connection header received
0807 22:55:00.041711 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:00.041716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:00.041864 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:00.041993 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:00.043253 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.044519 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.046117 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.046839 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.046981 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:00.048392 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:00.048400 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:00.048408 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:00.048433 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:00.048486 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:00.048504 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:00.048600 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:00.048804 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 22:55:00.048917 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":245,"threads_started":1}
I20260807 22:55:00.050191 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:00.038086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41745 (local address 127.9.194.193:35255)
0807 22:55:00.039151 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:00.039198 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:00.041424 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:00.042188 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:00.042196 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:00.042363 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:00.043075 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:00.043093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.044698 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.044702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:00.045942 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:00.045954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.047068 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.047073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:00.047279 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:00.048069 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:00.048072 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:00.048111 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:00.048191 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:00.048601 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:00.048609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:00.048614 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:55:00.048751 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 22:55:00.049071 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":338,"threads_started":1}
I20260807 22:55:00.051851 30327 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41745
I20260807 22:55:00.052160 30327 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:00.052889 30327 heartbeater.cc:511] Master 127.9.194.254:41745 requested a full tablet report, sending...
I20260807 22:55:00.054981 30126 ts_manager.cc:194] Registered new tserver with Master: 864fd3ecdc8d485b8867449e58a48791 (127.9.194.193:34507)
I20260807 22:55:00.056505 30126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.193:35255
W20260807 22:55:00.383137 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:00.383482 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:00.383639 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:00.391175 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:00.391299 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:00.391522 30331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:55:00.401031 30331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:00.402298 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:00.404247 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:00.420645 30338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:00.421298 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:00.421429 30340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:00.421932 30331 server_base.cc:1034] running on GCE node
I20260807 22:55:00.422721 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:00.424019 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:00.425297 30331 hybrid_clock.cc:648] HybridClock initialized: now 1786143300425226 us; error 50 us; skew 500 ppm
I20260807 22:55:00.425779 30331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:00.428915 30331 webserver.cc:460] Webserver started at http://127.9.194.194:34667/ using document root <none> and password file <none>
I20260807 22:55:00.430009 30331 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:00.430151 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:00.430578 30331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:00.435117 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "1da62e37977a428dbb8883faa97650e6"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "5343ca33cd83b7608a46d6068e1f6a23"
server_key_iv: "8a46454d35c9f3c312345ab78d4c7d15"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.436268 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "1da62e37977a428dbb8883faa97650e6"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "5343ca33cd83b7608a46d6068e1f6a23"
server_key_iv: "8a46454d35c9f3c312345ab78d4c7d15"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.443629 30331 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260807 22:55:00.448683 30348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.450416 30331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:00.450608 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "1da62e37977a428dbb8883faa97650e6"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "5343ca33cd83b7608a46d6068e1f6a23"
server_key_iv: "8a46454d35c9f3c312345ab78d4c7d15"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.450904 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:00.492627 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:00.493788 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:00.494146 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:00.495537 30331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:00.497814 30331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:00.497938 30331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.498075 30331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:00.498162 30331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.537205 30331 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:42059
I20260807 22:55:00.537218 30469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:42059 every 8 connection(s)
I20260807 22:55:00.539048 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:55:00.545855 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30331
I20260807 22:55:00.546085 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:55:00.546626 9995 external_mini_cluster.cc:1428] Setting key 7969e019e7a99d4aa06cfc2ca4354009
I20260807 22:55:00.550150 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--builtin_ntp_servers=127.9.194.212:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:55:00.553737 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:00.541919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:47983 (local address 127.9.194.254:41745)
0807 22:55:00.542099 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:00.542110 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:00.543951 (+ 1841us) server_negotiation.cc:407] Connection header received
0807 22:55:00.545899 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:00.545909 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:00.546017 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:00.546143 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:00.547826 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.548873 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.550934 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.551382 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.551545 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:00.552939 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:00.552944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:00.552947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:00.552964 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:00.552995 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:00.553006 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:00.553102 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:00.553324 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 22:55:00.553431 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:55:00.554841 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:00.542740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41745 (local address 127.9.194.194:47983)
0807 22:55:00.543774 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:00.543819 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:00.545535 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:00.546395 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:00.546407 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:00.546534 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:00.547346 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:00.547365 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.549044 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.549049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:00.550677 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:00.550730 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:00.551676 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:00.551683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:00.551908 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:00.552615 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:00.552618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:00.552661 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:00.552737 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:00.553104 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:00.553117 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:00.553127 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:55:00.553318 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 22:55:00.553702 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":321,"threads_started":1}
I20260807 22:55:00.556336 30470 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41745
I20260807 22:55:00.556679 30470 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:00.557489 30470 heartbeater.cc:511] Master 127.9.194.254:41745 requested a full tablet report, sending...
I20260807 22:55:00.558857 30126 ts_manager.cc:194] Registered new tserver with Master: 1da62e37977a428dbb8883faa97650e6 (127.9.194.194:42059)
I20260807 22:55:00.559726 30126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.194:47983
W20260807 22:55:00.894052 30473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:00.894322 30473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:00.894466 30473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:00.902050 30473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:00.902149 30473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:00.902326 30473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:55:00.911351 30473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:41745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:00.912848 30473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:00.914753 30473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:00.928016 30480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:00.927951 30483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:00.928068 30481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:00.928859 30473 server_base.cc:1034] running on GCE node
I20260807 22:55:00.929467 30473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:00.930406 30473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:00.931620 30473 hybrid_clock.cc:648] HybridClock initialized: now 1786143300931579 us; error 55 us; skew 500 ppm
I20260807 22:55:00.931931 30473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:00.934070 30473 webserver.cc:460] Webserver started at http://127.9.194.195:35905/ using document root <none> and password file <none>
I20260807 22:55:00.934767 30473 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:00.934895 30473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:00.935215 30473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:00.938071 30473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "e169ee639a96415bad1672bbc2860f4b"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "efb62fe412ffb4678f63b5db44e0f01c"
server_key_iv: "53478e1a317d973645acf548e412961d"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.938839 30473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "e169ee639a96415bad1672bbc2860f4b"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "efb62fe412ffb4678f63b5db44e0f01c"
server_key_iv: "53478e1a317d973645acf548e412961d"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.943830 30473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:55:00.947218 30490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.948638 30473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:00.948823 30473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "e169ee639a96415bad1672bbc2860f4b"
format_stamp: "Formatted at 2026-08-07 22:55:00 on dist-test-slave-5k54"
server_key: "efb62fe412ffb4678f63b5db44e0f01c"
server_key_iv: "53478e1a317d973645acf548e412961d"
server_key_version: "encryptionkey@0"
I20260807 22:55:00.949028 30473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:00.968047 30473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:00.968820 30473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:00.969126 30473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:00.970058 30473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:00.971552 30473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:00.971671 30473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:00.971807 30473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:00.971891 30473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:01.005649 30473 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:43563
I20260807 22:55:01.005704 30616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:43563 every 8 connection(s)
I20260807 22:55:01.007030 30473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:55:01.013887 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30473
I20260807 22:55:01.014065 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:55:01.014528 9995 external_mini_cluster.cc:1428] Setting key c59c05ce38d59e4da5499ff16ecada36
I20260807 22:55:01.020341 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.009681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:42477 (local address 127.9.194.254:41745)
0807 22:55:01.009914 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.009919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.011285 (+ 1366us) server_negotiation.cc:407] Connection header received
0807 22:55:01.013328 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.013334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.013431 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.013571 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:01.014744 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.015745 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.017659 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.018099 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.018231 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.019549 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:01.019554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:01.019557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:01.019574 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:01.019606 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:01.019617 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:01.019715 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.019928 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.020023 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:55:01.021310 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.010208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:41745 (local address 127.9.194.195:42477)
0807 22:55:01.011122 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.011164 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.013065 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.013771 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.013779 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.013876 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:01.014544 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.014561 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.015906 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.015912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.017506 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.017518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.018326 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.018334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.018554 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.019213 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:01.019215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:01.019251 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:01.019340 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:01.019714 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:01.019725 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:01.019735 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.019919 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.020282 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":273,"threads_started":1}
I20260807 22:55:01.022614 30617 heartbeater.cc:347] Connected to a master server at 127.9.194.254:41745
I20260807 22:55:01.022921 30617 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:01.023720 30617 heartbeater.cc:511] Master 127.9.194.254:41745 requested a full tablet report, sending...
I20260807 22:55:01.025099 30126 ts_manager.cc:194] Registered new tserver with Master: e169ee639a96415bad1672bbc2860f4b (127.9.194.195:43563)
I20260807 22:55:01.025899 30126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.195:42477
I20260807 22:55:01.029651 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:55:01.040271 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.033881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.9.194.254:41745)
0807 22:55:01.034049 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.034053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.034502 (+ 449us) server_negotiation.cc:407] Connection header received
0807 22:55:01.034702 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.034708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.034791 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.034923 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:01.035405 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.036153 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.037649 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.038075 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.038210 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.039489 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:01.039503 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:01.039507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:01.039526 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:01.039558 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:01.039588 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:01.039754 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.039953 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.040011 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:55:01.044422 30126 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:55:01.051644 30126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:55:01.059604 30327 heartbeater.cc:503] Master 127.9.194.254:41745 was elected leader, sending a full tablet report...
I20260807 22:55:01.101519 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.064634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43563 (local address 127.0.0.1:52788)
0807 22:55:01.091612 (+ 26978us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.091632 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.091752 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.092785 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.092789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.092817 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:01.093626 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.093638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.095733 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.095739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.098326 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.098339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.100624 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.100632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.100836 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100873 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.100970 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.101107 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26098,"mutex_wait_us":190,"thread_start_us":25991,"threads_started":1}
I20260807 22:55:01.101528 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.065208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.9.194.195:43563)
0807 22:55:01.090180 (+ 24972us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.090192 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.091876 (+ 1684us) server_negotiation.cc:407] Connection header received
0807 22:55:01.092001 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.092011 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.092281 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.092469 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:01.093812 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.095587 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.098514 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.100420 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.100580 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100871 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.101033 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.101206 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24774,"thread_start_us":24994,"threads_started":1}
I20260807 22:55:01.101562 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.065235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.9.194.193:34507)
0807 22:55:01.089429 (+ 24194us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.089443 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.091798 (+ 2355us) server_negotiation.cc:407] Connection header received
0807 22:55:01.092122 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.092132 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.092370 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.092581 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:01.093941 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.095609 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.097897 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.099646 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.099790 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100117 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.100986 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.101174 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23984,"thread_start_us":23816,"threads_started":1}
I20260807 22:55:01.101766 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.064561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.9.194.194:42059)
0807 22:55:01.089755 (+ 25194us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.089766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.091523 (+ 1757us) server_negotiation.cc:407] Connection header received
0807 22:55:01.091654 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.091666 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.091977 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.092190 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:01.093345 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.094592 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.098333 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.099806 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.099938 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100151 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.101241 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.101421 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25026,"thread_start_us":24916,"threads_started":1}
I20260807 22:55:01.102425 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.064508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:42059 (local address 127.0.0.1:46130)
0807 22:55:01.091004 (+ 26496us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.091058 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.091322 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.092513 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.092523 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.092562 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:01.093097 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.093112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.094812 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.094817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.098105 (+ 3288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.098121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.100293 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.100299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.100539 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100664 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.100804 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.101037 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25708,"mutex_wait_us":71,"thread_start_us":26387,"threads_started":1}
I20260807 22:55:01.102463 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.064936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33156)
0807 22:55:01.091651 (+ 26715us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.091671 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.091909 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.092924 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.092927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.092952 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:01.093649 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.093668 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.095793 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.095797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.097671 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.097685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.100268 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.100281 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.100572 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.100671 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.100804 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.102107 (+ 1303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26570,"mutex_wait_us":1237,"thread_start_us":25643,"threads_started":1}
I20260807 22:55:01.105880 30401 tablet_service.cc:1467] Processing CreateTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 (DEFAULT_TABLE table=test-table [id=93adf706e1814919af900a6d881719fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:01.106127 30540 tablet_service.cc:1467] Processing CreateTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 (DEFAULT_TABLE table=test-table [id=93adf706e1814919af900a6d881719fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:01.107226 30401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:01.106931 30257 tablet_service.cc:1467] Processing CreateTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 (DEFAULT_TABLE table=test-table [id=93adf706e1814919af900a6d881719fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:01.107615 30540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:01.108379 30257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:01.120839 30635 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Bootstrap starting.
I20260807 22:55:01.123606 30637 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Bootstrap starting.
I20260807 22:55:01.125820 30636 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Bootstrap starting.
I20260807 22:55:01.126946 30635 tablet_bootstrap.cc:654] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:01.127514 30637 tablet_bootstrap.cc:654] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:01.128218 30635 log.cc:824] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:01.128806 30637 log.cc:824] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:01.129586 30636 tablet_bootstrap.cc:654] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:01.130535 30636 log.cc:824] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:01.131350 30635 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: No bootstrap required, opened a new log
I20260807 22:55:01.131683 30635 ts_tablet_manager.cc:1397] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 22:55:01.133525 30636 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: No bootstrap required, opened a new log
I20260807 22:55:01.133734 30636 ts_tablet_manager.cc:1397] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 22:55:01.137363 30637 tablet_bootstrap.cc:492] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: No bootstrap required, opened a new log
I20260807 22:55:01.137645 30637 ts_tablet_manager.cc:1397] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260807 22:55:01.138916 30635 raft_consensus.cc:348] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.139225 30635 raft_consensus.cc:374] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:01.139283 30635 raft_consensus.cc:729] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da62e37977a428dbb8883faa97650e6, State: Initialized, Role: FOLLOWER
I20260807 22:55:01.140015 30635 consensus_queue.cc:260] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.141335 30470 heartbeater.cc:503] Master 127.9.194.254:41745 was elected leader, sending a full tablet report...
I20260807 22:55:01.142019 30635 ts_tablet_manager.cc:1428] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 22:55:01.143682 30636 raft_consensus.cc:348] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.144125 30636 raft_consensus.cc:374] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:01.144232 30636 raft_consensus.cc:729] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864fd3ecdc8d485b8867449e58a48791, State: Initialized, Role: FOLLOWER
I20260807 22:55:01.144861 30636 consensus_queue.cc:260] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.146662 30636 ts_tablet_manager.cc:1428] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260807 22:55:01.149144 30637 raft_consensus.cc:348] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.149525 30637 raft_consensus.cc:374] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:01.149636 30637 raft_consensus.cc:729] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e169ee639a96415bad1672bbc2860f4b, State: Initialized, Role: FOLLOWER
I20260807 22:55:01.150189 30637 consensus_queue.cc:260] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.151153 30617 heartbeater.cc:503] Master 127.9.194.254:41745 was elected leader, sending a full tablet report...
I20260807 22:55:01.151788 30637 ts_tablet_manager.cc:1428] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260807 22:55:01.232468 30641 raft_consensus.cc:482] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:55:01.232843 30641 raft_consensus.cc:504] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.234568 30641 leader_election.cc:288] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e169ee639a96415bad1672bbc2860f4b (127.9.194.195:43563), 864fd3ecdc8d485b8867449e58a48791 (127.9.194.193:34507)
I20260807 22:55:01.241751 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.234964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:36215 (local address 127.9.194.195:43563)
0807 22:55:01.235215 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.235224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.236104 (+ 880us) server_negotiation.cc:407] Connection header received
0807 22:55:01.236350 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.236354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.236415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.236514 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:01.237143 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.237876 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.239974 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.240734 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.240922 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.240994 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.241342 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.241470 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:55:01.241761 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.235262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:37091 (local address 127.9.194.193:34507)
0807 22:55:01.235544 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.235553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.236268 (+ 715us) server_negotiation.cc:407] Connection header received
0807 22:55:01.236471 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.236476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.236548 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.236648 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:01.237283 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.237975 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.239962 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.240713 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.240891 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.240966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.241339 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.241481 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:55:01.241994 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.235009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:43563 (local address 127.9.194.194:36215)
0807 22:55:01.235980 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.236000 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.236189 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.236535 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.236539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.236563 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:01.236992 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.237002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.238019 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.238026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.239772 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.239784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.240933 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.240938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.241160 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.241198 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.241305 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.241466 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":134,"thread_start_us":534,"threads_started":1}
I20260807 22:55:01.242012 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.235070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.9.194.194:37091)
0807 22:55:01.236162 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.236177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:01.236318 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:01.236665 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:01.236669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:01.236707 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:01.237132 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.237140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.238113 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.238121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.239760 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:01.239772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.240929 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.240936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:01.241163 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.241200 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:55:01.241310 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 22:55:01.241472 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":141,"thread_start_us":676,"threads_started":1}
I20260807 22:55:01.243028 30564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a651abd4c2d4bc7ad8f7a22f4d7a543" candidate_uuid: "1da62e37977a428dbb8883faa97650e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e169ee639a96415bad1672bbc2860f4b" is_pre_election: true
I20260807 22:55:01.243183 30278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a651abd4c2d4bc7ad8f7a22f4d7a543" candidate_uuid: "1da62e37977a428dbb8883faa97650e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864fd3ecdc8d485b8867449e58a48791" is_pre_election: true
I20260807 22:55:01.243628 30564 raft_consensus.cc:2454] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da62e37977a428dbb8883faa97650e6 in term 0.
I20260807 22:55:01.243806 30278 raft_consensus.cc:2454] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da62e37977a428dbb8883faa97650e6 in term 0.
I20260807 22:55:01.244334 30349 leader_election.cc:302] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da62e37977a428dbb8883faa97650e6, e169ee639a96415bad1672bbc2860f4b; no voters:
I20260807 22:55:01.244722 30641 raft_consensus.cc:2781] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:55:01.244887 30641 raft_consensus.cc:482] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:55:01.245009 30641 raft_consensus.cc:3037] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:01.247138 30641 raft_consensus.cc:504] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.247797 30641 leader_election.cc:288] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [CANDIDATE]: Term 1 election: Requested vote from peers e169ee639a96415bad1672bbc2860f4b (127.9.194.195:43563), 864fd3ecdc8d485b8867449e58a48791 (127.9.194.193:34507)
I20260807 22:55:01.248224 30564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a651abd4c2d4bc7ad8f7a22f4d7a543" candidate_uuid: "1da62e37977a428dbb8883faa97650e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e169ee639a96415bad1672bbc2860f4b"
I20260807 22:55:01.248399 30564 raft_consensus.cc:3037] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:01.248415 30278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a651abd4c2d4bc7ad8f7a22f4d7a543" candidate_uuid: "1da62e37977a428dbb8883faa97650e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864fd3ecdc8d485b8867449e58a48791"
I20260807 22:55:01.248617 30278 raft_consensus.cc:3037] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:01.250658 30564 raft_consensus.cc:2454] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da62e37977a428dbb8883faa97650e6 in term 1.
I20260807 22:55:01.250819 30278 raft_consensus.cc:2454] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da62e37977a428dbb8883faa97650e6 in term 1.
I20260807 22:55:01.251045 30349 leader_election.cc:302] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da62e37977a428dbb8883faa97650e6, e169ee639a96415bad1672bbc2860f4b; no voters:
I20260807 22:55:01.251331 30641 raft_consensus.cc:2781] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:01.251838 30641 raft_consensus.cc:686] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 1 LEADER]: Becoming Leader. State: Replica: 1da62e37977a428dbb8883faa97650e6, State: Running, Role: LEADER
I20260807 22:55:01.252421 30641 consensus_queue.cc:237] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } }
I20260807 22:55:01.257110 30126 catalog_manager.cc:5495] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1da62e37977a428dbb8883faa97650e6 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "1da62e37977a428dbb8883faa97650e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1da62e37977a428dbb8883faa97650e6" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 42059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 } health_report { overall_health: UNKNOWN } } }
W20260807 22:55:01.258752 30618 tablet.cc:2369] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:55:01.287285 30328 tablet.cc:2369] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:55:01.291229 30471 tablet.cc:2369] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:01.304697 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.299575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.9.194.194:42059)
0807 22:55:01.299763 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.299768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.299787 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:55:01.299940 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.299943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.300030 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.300199 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:01.300598 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.301312 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.302623 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.303015 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.303108 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.303620 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:01.304079 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:01.304209 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.304343 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.304432 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:55:01.312969 30278 raft_consensus.cc:1261] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 1 FOLLOWER]: Refusing update from remote peer 1da62e37977a428dbb8883faa97650e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:01.312965 30564 raft_consensus.cc:1261] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 1 FOLLOWER]: Refusing update from remote peer 1da62e37977a428dbb8883faa97650e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:01.314102 30641 consensus_queue.cc:1035] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e169ee639a96415bad1672bbc2860f4b" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 43563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:01.314481 30646 consensus_queue.cc:1035] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "864fd3ecdc8d485b8867449e58a48791" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 34507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:01.342060 30125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:01.342412 30125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:01.346256 30125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c003564bd2804514bf9a6f43c9074332: Sending DeleteTablet for 3 replicas of tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543
I20260807 22:55:01.347312 30401 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:01 UTC) from {username='slave'} at 127.0.0.1:46130
I20260807 22:55:01.347357 30257 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:01 UTC) from {username='slave'} at 127.0.0.1:33156
I20260807 22:55:01.347357 30540 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:01 UTC) from {username='slave'} at 127.0.0.1:52788
I20260807 22:55:01.349572 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30189
I20260807 22:55:01.360375 30662 tablet_replica.cc:330] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: stopping tablet replica
I20260807 22:55:01.360615 30660 tablet_replica.cc:330] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: stopping tablet replica
I20260807 22:55:01.361263 30660 raft_consensus.cc:2229] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:55:01.361486 30662 raft_consensus.cc:2229] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:01.362005 30662 raft_consensus.cc:2258] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:01.362159 30660 raft_consensus.cc:2258] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:01.364926 30660 ts_tablet_manager.cc:1905] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:01.365413 30662 ts_tablet_manager.cc:1905] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 864fd3ecdc8d485b8867449e58a48791: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:01.367250 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.350996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.9.194.193:34507)
0807 22:55:01.351167 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.351174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.356395 (+ 5221us) server_negotiation.cc:407] Connection header received
0807 22:55:01.356614 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.356621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.356751 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.356871 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:01.358282 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.359540 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.362239 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.362931 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.363032 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.365437 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:01.365448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:01.365464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:01.366261 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:01.366337 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:01.366364 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:01.366456 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.366743 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.366865 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:55:01.368422 30322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:01.370270 30661 tablet_replica.cc:330] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: stopping tablet replica
I20260807 22:55:01.370826 30661 raft_consensus.cc:2229] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:01.371464 30661 raft_consensus.cc:2258] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:01.374204 30661 ts_tablet_manager.cc:1905] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:01.377686 30660 ts_tablet_manager.cc:1918] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:01.378043 30660 log.cc:1196] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/1a651abd4c2d4bc7ad8f7a22f4d7a543
I20260807 22:55:01.378722 30660 ts_tablet_manager.cc:1939] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P 1da62e37977a428dbb8883faa97650e6: Deleting consensus metadata
I20260807 22:55:01.380793 30109 catalog_manager.cc:4841] TS 1da62e37977a428dbb8883faa97650e6 (127.9.194.194:42059): tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 (table test-table [id=93adf706e1814919af900a6d881719fd]) successfully deleted
I20260807 22:55:01.382257 30661 ts_tablet_manager.cc:1918] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:01.382431 30661 log.cc:1196] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/1a651abd4c2d4bc7ad8f7a22f4d7a543
I20260807 22:55:01.382990 30661 ts_tablet_manager.cc:1939] T 1a651abd4c2d4bc7ad8f7a22f4d7a543 P e169ee639a96415bad1672bbc2860f4b: Deleting consensus metadata
I20260807 22:55:01.384884 30107 catalog_manager.cc:4841] TS e169ee639a96415bad1672bbc2860f4b (127.9.194.195:43563): tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543 (table test-table [id=93adf706e1814919af900a6d881719fd]) successfully deleted
I20260807 22:55:01.476300 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30189
W20260807 22:55:01.497080 30108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.194.193:34507 (error 108)
I20260807 22:55:01.497875 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30331
I20260807 22:55:01.499516 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.498886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33178)
0807 22:55:01.499184 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.499317 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260807 22:55:01.499778 30108 catalog_manager.cc:4584] TS 864fd3ecdc8d485b8867449e58a48791 (127.9.194.193:34507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a651abd4c2d4bc7ad8f7a22f4d7a543: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
I20260807 22:55:01.505378 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.498932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.9.194.194:42059)
0807 22:55:01.499150 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.499157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.499175 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:55:01.499236 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.499239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.499332 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.499486 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:01.499997 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.500817 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.502151 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.502591 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.502738 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.503887 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:01.503895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:01.503909 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:01.504545 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:01.504590 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:01.504609 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:01.504737 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.504881 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.504977 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:55:01.506094 30465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:01.553609 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.553110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33190)
0807 22:55:01.553307 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.553420 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:55:01.554745 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.554345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33196)
0807 22:55:01.554542 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.554623 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 22:55:01.615182 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.614665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33206)
0807 22:55:01.614929 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.615019 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 22:55:01.616292 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.615937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33214)
0807 22:55:01.616090 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.616171 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:55:01.620168 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30331
I20260807 22:55:01.641409 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30473
I20260807 22:55:01.648674 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.642170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.9.194.195:43563)
0807 22:55:01.642337 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:01.642342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:01.642361 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:55:01.642538 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:01.642543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:01.642623 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:01.642714 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:01.643051 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.643868 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.645344 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:01.645736 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:01.645826 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:01.647273 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:01.647285 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:01.647306 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:01.647974 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:01.648020 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:01.648040 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:01.648133 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:01.648263 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 22:55:01.648344 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:55:01.649391 30612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:01.697085 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.696488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33222)
0807 22:55:01.696751 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.696869 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:55:01.698210 30631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:01.697766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:34507 (local address 127.0.0.1:33232)
0807 22:55:01.697980 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:01.698071 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.193:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 22:55:01.738997 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30473
I20260807 22:55:01.757570 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30091
I20260807 22:55:01.758702 30159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:01.868275 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30091
2026-08-07T22:55:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2933 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:01 dist-test-slave-5k54 krb5kdc[30671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:01 dist-test-slave-5k54 krb5kdc[30671](info): set up 2 sockets
Aug 07 22:55:01 dist-test-slave-5k54 krb5kdc[30671](info): commencing operation
krb5kdc: starting...
W20260807 22:55:03.069008 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:03 dist-test-slave-5k54 krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:03.378520 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35781
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:35781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:03.722061 30685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:03.722342 30685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:03.722431 30685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:03.729971 30685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:03.730089 30685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:03.730139 30685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:03.730206 30685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:03.730252 30685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:03.740048 30685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:35781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:03.741263 30685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:03.742524 30685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:03.754319 30690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:03.754319 30691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:03.754793 30694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:03.755263 30685 server_base.cc:1034] running on GCE node
I20260807 22:55:03.756141 30685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:03.757625 30685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:03.758915 30685 hybrid_clock.cc:648] HybridClock initialized: now 1786143303758826 us; error 37 us; skew 500 ppm
I20260807 22:55:03.759269 30685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:03 dist-test-slave-5k54 krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:03.762552 30685 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:03.764118 30685 webserver.cc:460] Webserver started at http://127.9.194.254:46849/ using document root <none> and password file <none>
I20260807 22:55:03.764770 30685 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:03.764897 30685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:03.765167 30685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:03.768270 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6"
format_stamp: "Formatted at 2026-08-07 22:55:03 on dist-test-slave-5k54"
server_key: "1c4b59cda6259a48d7a2373f37bc564b"
server_key_iv: "8225656b36f692833c6b1f5ca8c34b71"
server_key_version: "encryptionkey@0"
I20260807 22:55:03.768981 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6"
format_stamp: "Formatted at 2026-08-07 22:55:03 on dist-test-slave-5k54"
server_key: "1c4b59cda6259a48d7a2373f37bc564b"
server_key_iv: "8225656b36f692833c6b1f5ca8c34b71"
server_key_version: "encryptionkey@0"
I20260807 22:55:03.774204 30685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:55:03.777707 30704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:03.779039 30685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:55:03.779207 30685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6"
format_stamp: "Formatted at 2026-08-07 22:55:03 on dist-test-slave-5k54"
server_key: "1c4b59cda6259a48d7a2373f37bc564b"
server_key_iv: "8225656b36f692833c6b1f5ca8c34b71"
server_key_version: "encryptionkey@0"
I20260807 22:55:03.779415 30685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:03.802446 30685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:03.807142 30685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:03.807502 30685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:03.826449 30685 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:35781
I20260807 22:55:03.826483 30762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:35781 every 8 connection(s)
I20260807 22:55:03.827903 30685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:03.828334 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30685
I20260807 22:55:03.828542 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:03.829208 9995 external_mini_cluster.cc:1428] Setting key 366173e78c0fb062fd881d151d967c61
I20260807 22:55:03.832767 30763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:55:03 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:03.848320 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: Bootstrap starting.
I20260807 22:55:03.851687 30763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:03.852967 30763 log.cc:824] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:03.856101 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: No bootstrap required, opened a new log
I20260807 22:55:03.856629 30766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:03.831185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.9.194.254:35781)
0807 22:55:03.832007 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:03.832021 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:03.832069 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:55:03.834498 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:03.834525 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:03.835023 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:03.835477 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:03.836836 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:03.838934 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:03.841410 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:03.842637 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:03.842826 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:03.848861 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:03.848870 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:03.848890 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:03.848931 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:03.852257 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:03.853064 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:03.853073 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:03.853081 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:03.853308 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:03.853748 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:03.853753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:03.853757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:03.854324 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:03.854683 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:03.855147 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 22:55:03.855603 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":368,"threads_started":1}
I20260807 22:55:03.864424 30763 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } }
I20260807 22:55:03.864802 30763 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:03.864912 30763 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc2553c7d34a4faeb7adedfa98dbc0b6, State: Initialized, Role: FOLLOWER
I20260807 22:55:03.865562 30763 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } }
I20260807 22:55:03.865859 30763 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:03.866096 30763 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:03.866281 30763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:03.868480 30763 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } }
I20260807 22:55:03.868911 30763 leader_election.cc:302] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc2553c7d34a4faeb7adedfa98dbc0b6; no voters:
I20260807 22:55:03.869532 30763 leader_election.cc:288] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:03.869684 30768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:03.870391 30768 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [term 1 LEADER]: Becoming Leader. State: Replica: cc2553c7d34a4faeb7adedfa98dbc0b6, State: Running, Role: LEADER
I20260807 22:55:03.870966 30768 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } }
I20260807 22:55:03.871650 30763 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:03.874241 30770 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2553c7d34a4faeb7adedfa98dbc0b6. Latest consensus state: current_term: 1 leader_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } } }
I20260807 22:55:03.874650 30770 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:03.874495 30769 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2553c7d34a4faeb7adedfa98dbc0b6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35781 } } }
I20260807 22:55:03.874759 30769 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:03.876607 30777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:03.882117 30777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:03.891849 30777 catalog_manager.cc:1347] Generated new cluster ID: 83fab5a50b1546df93b64037e8c5de80
I20260807 22:55:03.891978 30777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:03.908337 30777 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:03.909638 30777 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:03.917210 30777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: Generated new TSK 0
I20260807 22:55:03.918061 30777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.194.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
Entry for principal kudu/127.9.194.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab.
I20260807 22:55:03.988452 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:04.328999 30791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:04.329278 30791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:04.329461 30791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:04.336742 30791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:04.336846 30791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:04.337070 30791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:55:04.346922 30791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.193.keytab
--principal=kudu/127.9.194.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:04.348228 30791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:04.349581 30791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:04.363220 30796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:04.363263 30797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:04.363346 30799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:04.364132 30791 server_base.cc:1034] running on GCE node
I20260807 22:55:04.364845 30791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:04.365885 30791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:04.367133 30791 hybrid_clock.cc:648] HybridClock initialized: now 1786143304367092 us; error 52 us; skew 500 ppm
I20260807 22:55:04.367542 30791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:04 dist-test-slave-5k54 krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143304, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:04.370913 30791 init.cc:377] Logged in from keytab as kudu/127.9.194.193@KRBTEST.COM (short username kudu)
I20260807 22:55:04.372503 30791 webserver.cc:460] Webserver started at http://127.9.194.193:34921/ using document root <none> and password file <none>
I20260807 22:55:04.373322 30791 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:04.373484 30791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:04.373821 30791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:04.377143 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "87eb97726e35412b95ecf9d7e204a113"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "41c7a2f3c8859f6cc121b4faeaeef264"
server_key_iv: "6ec86220c35ca23fd50bfcb4528e7260"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.377996 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "87eb97726e35412b95ecf9d7e204a113"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "41c7a2f3c8859f6cc121b4faeaeef264"
server_key_iv: "6ec86220c35ca23fd50bfcb4528e7260"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.383746 30791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 22:55:04.387295 30807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.388666 30791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:04.388846 30791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "87eb97726e35412b95ecf9d7e204a113"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "41c7a2f3c8859f6cc121b4faeaeef264"
server_key_iv: "6ec86220c35ca23fd50bfcb4528e7260"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.389062 30791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:04.418936 30791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:04.423586 30791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:04.423930 30791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:04.425038 30791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:04.426520 30791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:04.426640 30791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.426800 30791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:04.426900 30791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.463017 30791 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:41165
I20260807 22:55:04.463068 30933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:41165 every 8 connection(s)
I20260807 22:55:04.464460 30791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:55:04.468360 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30791
I20260807 22:55:04.468552 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:55:04.469151 9995 external_mini_cluster.cc:1428] Setting key 6bed88d9e2afb546eb0b9ed0c0c4d84e
Aug 07 22:55:04 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143304, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.193@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:04.488123 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:04.467140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:34859 (local address 127.9.194.254:35781)
0807 22:55:04.467652 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:04.467656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:04.468713 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 22:55:04.471033 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:04.471039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:04.471134 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:04.471250 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:04.473453 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:04.474232 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:04.476680 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:04.477288 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:04.477421 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:04.481724 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:04.481730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:04.481733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:04.481770 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:04.484747 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:04.485550 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:04.485561 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:04.485564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:04.485742 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:04.486231 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:04.486241 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:04.486244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:04.486527 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:04.486698 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:04.487498 (+ 800us) server_negotiation.cc:299] Negotiation successful
0807 22:55:04.487741 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":297,"threads_started":1}
I20260807 22:55:04.489223 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:04.467665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:35781 (local address 127.9.194.193:34859)
0807 22:55:04.468544 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:04.468601 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:04.470688 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:04.471600 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:04.471611 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:04.472436 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:04.473268 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:04.473295 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:04.474398 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:04.474406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:04.476521 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:04.476539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:04.477543 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:04.477553 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:04.477883 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:04.478656 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:04.478688 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:04.481484 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:04.484916 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:04.484934 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:04.484951 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:04.485408 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:04.485866 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:04.485872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:04.485876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:04.486097 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:04.486667 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:04.486676 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:04.487045 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 22:55:04.487320 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 22:55:04.487930 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":339,"threads_started":1}
I20260807 22:55:04.490967 30934 heartbeater.cc:347] Connected to a master server at 127.9.194.254:35781
I20260807 22:55:04.491283 30934 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:04.492228 30934 heartbeater.cc:511] Master 127.9.194.254:35781 requested a full tablet report, sending...
I20260807 22:55:04.494717 30722 ts_manager.cc:194] Registered new tserver with Master: 87eb97726e35412b95ecf9d7e204a113 (127.9.194.193:41165)
I20260807 22:55:04.496884 30722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.193@KRBTEST.COM'} at 127.9.194.193:34859
WARNING: no policy specified for kudu/127.9.194.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
Entry for principal kudu/127.9.194.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab.
I20260807 22:55:04.539077 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:04.872637 30941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:04.873037 30941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:04.873224 30941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:04.882031 30941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:04.882180 30941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:04.882354 30941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:55:04.892464 30941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.194.keytab
--principal=kudu/127.9.194.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:04.893756 30941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:04.895084 30941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:04.908370 30947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:04.908473 30950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:04.908984 30946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:04.909250 30941 server_base.cc:1034] running on GCE node
I20260807 22:55:04.909894 30941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:04.910854 30941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:04.912076 30941 hybrid_clock.cc:648] HybridClock initialized: now 1786143304912010 us; error 53 us; skew 500 ppm
I20260807 22:55:04.912411 30941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:04 dist-test-slave-5k54 krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143304, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:04.915598 30941 init.cc:377] Logged in from keytab as kudu/127.9.194.194@KRBTEST.COM (short username kudu)
I20260807 22:55:04.917017 30941 webserver.cc:460] Webserver started at http://127.9.194.194:41399/ using document root <none> and password file <none>
I20260807 22:55:04.917721 30941 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:04.917873 30941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:04.918191 30941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:04.921108 30941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "4568699fc7ca493d9755bb0af11c1881"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "99fc78ba156b95aba85ed3c93d5d6042"
server_key_iv: "82a5949873e86c3f8a576a249fd22951"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.921895 30941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "4568699fc7ca493d9755bb0af11c1881"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "99fc78ba156b95aba85ed3c93d5d6042"
server_key_iv: "82a5949873e86c3f8a576a249fd22951"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.926993 30941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:04.930480 30958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.931819 30941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:04.931993 30941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "4568699fc7ca493d9755bb0af11c1881"
format_stamp: "Formatted at 2026-08-07 22:55:04 on dist-test-slave-5k54"
server_key: "99fc78ba156b95aba85ed3c93d5d6042"
server_key_iv: "82a5949873e86c3f8a576a249fd22951"
server_key_version: "encryptionkey@0"
I20260807 22:55:04.932191 30941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:04.953624 30941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:04.957782 30941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:04.958070 30941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:04.959033 30941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:04.960356 30941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:04.960484 30941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.960633 30941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:04.960705 30941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:04.994313 30941 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:38943
I20260807 22:55:04.994350 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:38943 every 8 connection(s)
I20260807 22:55:04.995661 30941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:55:04.998344 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 30941
I20260807 22:55:04.998558 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:55:04.999089 9995 external_mini_cluster.cc:1428] Setting key b3d652903f41bf818274f9e317774a68
Aug 07 22:55:05 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143304, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.194@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:05.017249 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:04.998096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:60503 (local address 127.9.194.254:35781)
0807 22:55:04.998551 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:04.998556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:04.999898 (+ 1342us) server_negotiation.cc:407] Connection header received
0807 22:55:05.002151 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.002156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.002243 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.002374 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:05.003973 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.004707 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.006383 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.006818 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.006973 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.010688 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:05.010697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:05.010704 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:05.010749 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:05.013991 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.014972 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.014975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.014977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.015105 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.015610 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.015613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.015615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.015861 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:05.015989 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.016819 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.016993 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":242,"threads_started":1}
I20260807 22:55:05.018975 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:04.998796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:35781 (local address 127.9.194.194:60503)
0807 22:55:04.999687 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:04.999759 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.001884 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.002613 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.002621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.003154 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:05.003800 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.003818 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.004887 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.004892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.006244 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.006256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.007065 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.007072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.007299 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.008049 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:05.008071 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:05.010457 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:05.014130 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:05.014140 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:05.014157 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:05.014847 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:05.015212 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:05.015220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:05.015227 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:05.015501 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:05.015984 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:05.015995 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:05.016440 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.016737 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.017323 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":360,"threads_started":1}
I20260807 22:55:05.020747 31081 heartbeater.cc:347] Connected to a master server at 127.9.194.254:35781
I20260807 22:55:05.021065 31081 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:05.021765 31081 heartbeater.cc:511] Master 127.9.194.254:35781 requested a full tablet report, sending...
I20260807 22:55:05.023290 30722 ts_manager.cc:194] Registered new tserver with Master: 4568699fc7ca493d9755bb0af11c1881 (127.9.194.194:38943)
I20260807 22:55:05.024605 30722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.194@KRBTEST.COM'} at 127.9.194.194:60503
WARNING: no policy specified for kudu/127.9.194.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
Entry for principal kudu/127.9.194.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab.
I20260807 22:55:05.068905 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:05.403097 31088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:05.403370 31088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:05.403542 31088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:05.410764 31088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:05.410866 31088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:05.410981 31088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:55:05.420518 31088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.195.keytab
--principal=kudu/127.9.194.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:05.421726 31088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:05.423017 31088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:05.436497 31095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:05.436446 31098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:05.436532 31093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:05.437501 31088 server_base.cc:1034] running on GCE node
I20260807 22:55:05.438063 31088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:05.438957 31088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:05.440191 31088 hybrid_clock.cc:648] HybridClock initialized: now 1786143305440143 us; error 35 us; skew 500 ppm
I20260807 22:55:05.440558 31088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:05 dist-test-slave-5k54 krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:05.443894 31088 init.cc:377] Logged in from keytab as kudu/127.9.194.195@KRBTEST.COM (short username kudu)
I20260807 22:55:05.445367 31088 webserver.cc:460] Webserver started at http://127.9.194.195:35235/ using document root <none> and password file <none>
I20260807 22:55:05.446062 31088 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:05.446208 31088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:05.446532 31088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:05.449501 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "1f0820ed2eef4e8fae55e03c4cedadd7"
format_stamp: "Formatted at 2026-08-07 22:55:05 on dist-test-slave-5k54"
server_key: "e91d9a325257870dd8d065b1d7ffceca"
server_key_iv: "7a615ff3df9082a0190f3243cdd24b28"
server_key_version: "encryptionkey@0"
I20260807 22:55:05.450519 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "1f0820ed2eef4e8fae55e03c4cedadd7"
format_stamp: "Formatted at 2026-08-07 22:55:05 on dist-test-slave-5k54"
server_key: "e91d9a325257870dd8d065b1d7ffceca"
server_key_iv: "7a615ff3df9082a0190f3243cdd24b28"
server_key_version: "encryptionkey@0"
I20260807 22:55:05.455935 31088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:55:05.459476 31107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:05.460781 31088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:05.460983 31088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "1f0820ed2eef4e8fae55e03c4cedadd7"
format_stamp: "Formatted at 2026-08-07 22:55:05 on dist-test-slave-5k54"
server_key: "e91d9a325257870dd8d065b1d7ffceca"
server_key_iv: "7a615ff3df9082a0190f3243cdd24b28"
server_key_version: "encryptionkey@0"
I20260807 22:55:05.461186 31088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:05.485287 31088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:05.489856 31088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:05.490154 31088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:05.491142 31088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:05.492478 31088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:05.492581 31088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:05.492731 31088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:05.492805 31088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:05.500825 30934 heartbeater.cc:503] Master 127.9.194.254:35781 was elected leader, sending a full tablet report...
I20260807 22:55:05.526669 31088 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:39053
I20260807 22:55:05.526719 31229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:39053 every 8 connection(s)
I20260807 22:55:05.528071 31088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:55:05.528659 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31088
I20260807 22:55:05.528854 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:55:05.529425 9995 external_mini_cluster.cc:1428] Setting key c337b018787dad27f2fa4f9bfdd5e4e0
Aug 07 22:55:05 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.195@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:05.548539 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.530946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:57291 (local address 127.9.194.254:35781)
0807 22:55:05.531497 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.531503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.532702 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 22:55:05.534521 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.534525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.534602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.534690 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:05.536215 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.537094 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.538581 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.539166 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.539263 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.542572 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:05.542577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:05.542580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:05.542613 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:05.545847 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.546557 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.546561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.546563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.546682 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.547100 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.547102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.547104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.547343 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:05.547526 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.548113 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.548266 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":273,"threads_started":1}
I20260807 22:55:05.549475 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.531603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:35781 (local address 127.9.194.195:57291)
0807 22:55:05.532517 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.532574 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.534274 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.534892 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.534902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.535498 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:05.536064 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.536080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.537234 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.537238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.538458 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.538470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.539348 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.539353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.539616 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.540233 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:05.540254 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:05.542389 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:05.545986 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:05.545993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:05.546004 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:05.546443 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:05.546794 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:05.546798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:05.546801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:05.546996 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:05.547503 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:05.547508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:05.547789 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.548010 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.548366 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":359,"threads_started":1}
I20260807 22:55:05.550823 31230 heartbeater.cc:347] Connected to a master server at 127.9.194.254:35781
I20260807 22:55:05.551139 31230 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:05.551879 31230 heartbeater.cc:511] Master 127.9.194.254:35781 requested a full tablet report, sending...
I20260807 22:55:05.553311 30722 ts_manager.cc:194] Registered new tserver with Master: 1f0820ed2eef4e8fae55e03c4cedadd7 (127.9.194.195:39053)
I20260807 22:55:05.554136 30722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.194.195@KRBTEST.COM'} at 127.9.194.195:57291
I20260807 22:55:05.557535 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:55:05.574313 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.561761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.9.194.254:35781)
0807 22:55:05.561927 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.561931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.562381 (+ 450us) server_negotiation.cc:407] Connection header received
0807 22:55:05.562532 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.562537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.562628 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.562732 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:05.564224 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.565006 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.566476 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.566894 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.566986 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.569388 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:05.569392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:05.569395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:05.569427 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:05.571916 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.572619 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.572623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.572626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.572744 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.573155 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.573159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.573161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.573464 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:05.573649 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.573900 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.574097 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:55:05.578114 30722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:55:05.585206 30722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:55:05.607553 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.598010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.9.194.194:38943)
0807 22:55:05.598864 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.598874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.599308 (+ 434us) server_negotiation.cc:407] Connection header received
0807 22:55:05.600219 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.600230 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.600495 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.601340 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:05.601551 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.602560 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.605316 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.606268 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.606419 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.606649 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.607012 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.607189 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":544,"threads_started":1}
I20260807 22:55:05.607627 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.597430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:38943 (local address 127.0.0.1:37990)
0807 22:55:05.599177 (+ 1747us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.599195 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.599314 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.600756 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.600764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.600790 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:05.601229 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.601246 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.602698 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.602707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.605164 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.605177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.606512 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.606520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.606749 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.606810 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.606947 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.607140 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1634,"mutex_wait_us":21,"thread_start_us":1493,"threads_started":1}
I20260807 22:55:05.609498 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.597299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39053 (local address 127.0.0.1:47508)
0807 22:55:05.598048 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.598087 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.598245 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.601059 (+ 2814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.601066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.601106 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:05.601752 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.601769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.604106 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.604114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.606437 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.606451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.608623 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.608630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.608820 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.608856 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.608971 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.609107 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":1,"thread_start_us":916,"threads_started":1}
I20260807 22:55:05.609895 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.597520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.9.194.195:39053)
0807 22:55:05.599953 (+ 2433us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.599963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.599995 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:55:05.600089 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.600097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.600778 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.602011 (+ 1233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:05.602364 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.603744 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.606656 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.608414 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.608552 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.608784 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.609124 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.609297 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2237,"thread_start_us":2077,"threads_started":1}
I20260807 22:55:05.611953 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.599715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:41165 (local address 127.0.0.1:59416)
0807 22:55:05.600238 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.600256 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.600357 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.603793 (+ 3436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.603800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.603832 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:05.604405 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.604418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.606344 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.606351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.608477 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.608490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.611181 (+ 2691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.611188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.611367 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.611398 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.611525 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.611660 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":286,"threads_started":1}
I20260807 22:55:05.612036 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.600063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.9.194.193:41165)
0807 22:55:05.603198 (+ 3135us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.603208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.603234 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:55:05.603328 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.603334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.603594 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.603781 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:05.604603 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.606184 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.609546 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.610962 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.611107 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.611322 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.611561 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.611718 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2972,"thread_start_us":2789,"threads_started":1}
I20260807 22:55:05.612504 31010 tablet_service.cc:1467] Processing CreateTablet for tablet c07a18b1450347ada07b533cbc9a31b4 (DEFAULT_TABLE table=test-table [id=4115dcab5a9e4853a583f1544093f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:05.613250 31161 tablet_service.cc:1467] Processing CreateTablet for tablet c07a18b1450347ada07b533cbc9a31b4 (DEFAULT_TABLE table=test-table [id=4115dcab5a9e4853a583f1544093f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:05.614276 31010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a18b1450347ada07b533cbc9a31b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:05.614684 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a18b1450347ada07b533cbc9a31b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:05.615460 30860 tablet_service.cc:1467] Processing CreateTablet for tablet c07a18b1450347ada07b533cbc9a31b4 (DEFAULT_TABLE table=test-table [id=4115dcab5a9e4853a583f1544093f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:05.616787 30860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07a18b1450347ada07b533cbc9a31b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:05.625033 31253 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Bootstrap starting.
I20260807 22:55:05.629036 31254 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Bootstrap starting.
I20260807 22:55:05.629997 31253 tablet_bootstrap.cc:654] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:05.631585 31253 log.cc:824] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:05.632102 31255 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Bootstrap starting.
I20260807 22:55:05.633159 31254 tablet_bootstrap.cc:654] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:05.634404 31254 log.cc:824] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:05.635190 31255 tablet_bootstrap.cc:654] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:05.636361 31253 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: No bootstrap required, opened a new log
I20260807 22:55:05.636435 31255 log.cc:824] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:05.636708 31253 ts_tablet_manager.cc:1397] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260807 22:55:05.638314 31254 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: No bootstrap required, opened a new log
I20260807 22:55:05.638576 31254 ts_tablet_manager.cc:1397] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 22:55:05.639550 31255 tablet_bootstrap.cc:492] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: No bootstrap required, opened a new log
I20260807 22:55:05.639798 31255 ts_tablet_manager.cc:1397] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 22:55:05.644870 31253 raft_consensus.cc:348] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.645260 31253 raft_consensus.cc:374] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:05.645367 31253 raft_consensus.cc:729] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4568699fc7ca493d9755bb0af11c1881, State: Initialized, Role: FOLLOWER
I20260807 22:55:05.646205 31253 consensus_queue.cc:260] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.647766 31081 heartbeater.cc:503] Master 127.9.194.254:35781 was elected leader, sending a full tablet report...
I20260807 22:55:05.648505 31253 ts_tablet_manager.cc:1428] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260807 22:55:05.649120 31254 raft_consensus.cc:348] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.649536 31254 raft_consensus.cc:374] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:05.649654 31254 raft_consensus.cc:729] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0820ed2eef4e8fae55e03c4cedadd7, State: Initialized, Role: FOLLOWER
I20260807 22:55:05.650262 31254 consensus_queue.cc:260] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.650200 31255 raft_consensus.cc:348] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.650609 31255 raft_consensus.cc:374] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:05.650727 31255 raft_consensus.cc:729] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87eb97726e35412b95ecf9d7e204a113, State: Initialized, Role: FOLLOWER
I20260807 22:55:05.651541 31230 heartbeater.cc:503] Master 127.9.194.254:35781 was elected leader, sending a full tablet report...
I20260807 22:55:05.651463 31255 consensus_queue.cc:260] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.652251 31254 ts_tablet_manager.cc:1428] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260807 22:55:05.654304 31255 ts_tablet_manager.cc:1428] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 22:55:05.670569 31261 raft_consensus.cc:482] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:55:05.670842 31261 raft_consensus.cc:504] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.672714 31261 leader_election.cc:288] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0820ed2eef4e8fae55e03c4cedadd7 (127.9.194.195:39053), 4568699fc7ca493d9755bb0af11c1881 (127.9.194.194:38943)
I20260807 22:55:05.679541 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.673044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:44127 (local address 127.9.194.195:39053)
0807 22:55:05.673250 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.673258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.673839 (+ 581us) server_negotiation.cc:407] Connection header received
0807 22:55:05.674078 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.674087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.674145 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.674240 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:05.674829 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.675874 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.677825 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.678600 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.678784 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.678857 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.679136 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.679261 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:55:05.679868 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.673240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:34261 (local address 127.9.194.194:38943)
0807 22:55:05.673443 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.673452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.674172 (+ 720us) server_negotiation.cc:407] Connection header received
0807 22:55:05.674370 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.674374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.674476 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.674572 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:05.675247 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.676140 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.678063 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.678819 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.678995 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.679073 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.679345 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.679508 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:55:05.679910 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.672854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:39053 (local address 127.9.194.193:44127)
0807 22:55:05.673724 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.673740 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.673927 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.674260 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.674264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.674291 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:05.674673 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.674686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.675999 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.676006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.677678 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.677690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.678799 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.678804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.678973 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.679001 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.679110 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.679235 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":17,"thread_start_us":401,"threads_started":1}
I20260807 22:55:05.679982 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.673225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:38943 (local address 127.9.194.193:34261)
0807 22:55:05.674066 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:05.674080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:05.674224 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:05.674610 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:05.674618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:05.674651 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:05.675058 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.675071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.676307 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.676313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.677902 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:05.677914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.679043 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.679049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:05.679193 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.679216 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:55:05.679319 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:55:05.679460 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":25,"thread_start_us":521,"threads_started":1}
I20260807 22:55:05.680905 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a18b1450347ada07b533cbc9a31b4" candidate_uuid: "87eb97726e35412b95ecf9d7e204a113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" is_pre_election: true
I20260807 22:55:05.681015 31034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a18b1450347ada07b533cbc9a31b4" candidate_uuid: "87eb97726e35412b95ecf9d7e204a113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4568699fc7ca493d9755bb0af11c1881" is_pre_election: true
I20260807 22:55:05.681427 31181 raft_consensus.cc:2454] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87eb97726e35412b95ecf9d7e204a113 in term 0.
I20260807 22:55:05.681455 31034 raft_consensus.cc:2454] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87eb97726e35412b95ecf9d7e204a113 in term 0.
I20260807 22:55:05.682062 30809 leader_election.cc:302] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4568699fc7ca493d9755bb0af11c1881, 87eb97726e35412b95ecf9d7e204a113; no voters:
I20260807 22:55:05.682443 31261 raft_consensus.cc:2781] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:55:05.682618 31261 raft_consensus.cc:482] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:55:05.682711 31261 raft_consensus.cc:3037] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:05.685009 31261 raft_consensus.cc:504] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.685658 31261 leader_election.cc:288] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [CANDIDATE]: Term 1 election: Requested vote from peers 1f0820ed2eef4e8fae55e03c4cedadd7 (127.9.194.195:39053), 4568699fc7ca493d9755bb0af11c1881 (127.9.194.194:38943)
I20260807 22:55:05.686003 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a18b1450347ada07b533cbc9a31b4" candidate_uuid: "87eb97726e35412b95ecf9d7e204a113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7"
I20260807 22:55:05.686118 31034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07a18b1450347ada07b533cbc9a31b4" candidate_uuid: "87eb97726e35412b95ecf9d7e204a113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4568699fc7ca493d9755bb0af11c1881"
I20260807 22:55:05.686190 31181 raft_consensus.cc:3037] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:05.686337 31034 raft_consensus.cc:3037] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:05.688499 31181 raft_consensus.cc:2454] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87eb97726e35412b95ecf9d7e204a113 in term 1.
I20260807 22:55:05.688680 31034 raft_consensus.cc:2454] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87eb97726e35412b95ecf9d7e204a113 in term 1.
I20260807 22:55:05.688922 30810 leader_election.cc:302] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0820ed2eef4e8fae55e03c4cedadd7, 87eb97726e35412b95ecf9d7e204a113; no voters:
I20260807 22:55:05.689235 31261 raft_consensus.cc:2781] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:05.689694 31261 raft_consensus.cc:686] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 1 LEADER]: Becoming Leader. State: Replica: 87eb97726e35412b95ecf9d7e204a113, State: Running, Role: LEADER
I20260807 22:55:05.690161 31261 consensus_queue.cc:237] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } }
I20260807 22:55:05.694777 30721 catalog_manager.cc:5495] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87eb97726e35412b95ecf9d7e204a113 (127.9.194.193). New cstate: current_term: 1 leader_uuid: "87eb97726e35412b95ecf9d7e204a113" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87eb97726e35412b95ecf9d7e204a113" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 41165 } health_report { overall_health: HEALTHY } } }
W20260807 22:55:05.716920 30936 tablet.cc:2369] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:05.731204 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.725802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59430 (local address 127.9.194.193:41165)
0807 22:55:05.726015 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.726021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.726092 (+ 71us) server_negotiation.cc:407] Connection header received
0807 22:55:05.726349 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.726354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.726449 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.726600 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:05.727009 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.727748 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.729093 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.729501 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.729635 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.730122 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:05.730612 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:05.730768 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.730847 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.730934 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:55:05.739310 31034 raft_consensus.cc:1261] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 1 FOLLOWER]: Refusing update from remote peer 87eb97726e35412b95ecf9d7e204a113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:05.739454 31181 raft_consensus.cc:1261] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 1 FOLLOWER]: Refusing update from remote peer 87eb97726e35412b95ecf9d7e204a113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:05.740334 31261 consensus_queue.cc:1035] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4568699fc7ca493d9755bb0af11c1881" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:05.740643 31265 consensus_queue.cc:1035] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0820ed2eef4e8fae55e03c4cedadd7" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:55:05.747520 31082 tablet.cc:2369] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:05.758028 31270 mvcc.cc:205] Tried to move back new op lower bound from 7316042980298493952 to 7316042980111642624. Current Snapshot: MvccSnapshot[applied={T|T < 7316042980298493952 or (T in {7316042980298493952})}]
I20260807 22:55:05.765853 30722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:05.766140 30722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:05.769867 30722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc2553c7d34a4faeb7adedfa98dbc0b6: Sending DeleteTablet for 3 replicas of tablet c07a18b1450347ada07b533cbc9a31b4
I20260807 22:55:05.771236 31161 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a18b1450347ada07b533cbc9a31b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:05 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:47508
I20260807 22:55:05.771447 31010 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a18b1450347ada07b533cbc9a31b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:05 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:37990
I20260807 22:55:05.771986 30860 tablet_service.cc:1514] Processing DeleteTablet for tablet c07a18b1450347ada07b533cbc9a31b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:05 UTC) from {username='kudu', principal='kudu/127.9.194.254@KRBTEST.COM'} at 127.0.0.1:59416
I20260807 22:55:05.772358 31280 tablet_replica.cc:330] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: stopping tablet replica
I20260807 22:55:05.772682 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30791
I20260807 22:55:05.772771 31280 raft_consensus.cc:2229] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:05.773314 31280 raft_consensus.cc:2258] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:05.775347 31280 ts_tablet_manager.cc:1905] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:05.782513 31280 ts_tablet_manager.cc:1918] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:05.782689 31280 log.cc:1196] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/c07a18b1450347ada07b533cbc9a31b4
I20260807 22:55:05.783293 31280 ts_tablet_manager.cc:1939] T c07a18b1450347ada07b533cbc9a31b4 P 4568699fc7ca493d9755bb0af11c1881: Deleting consensus metadata
I20260807 22:55:05.785296 30706 catalog_manager.cc:4841] TS 4568699fc7ca493d9755bb0af11c1881 (127.9.194.194:38943): tablet c07a18b1450347ada07b533cbc9a31b4 (table test-table [id=4115dcab5a9e4853a583f1544093f9da]) successfully deleted
I20260807 22:55:05.786059 31281 tablet_replica.cc:330] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: stopping tablet replica
I20260807 22:55:05.786580 31281 raft_consensus.cc:2229] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:55:05.787451 31281 raft_consensus.cc:2258] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:05.790138 31281 ts_tablet_manager.cc:1905] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:05.791031 31279 tablet_replica.cc:330] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: stopping tablet replica
I20260807 22:55:05.791464 31279 raft_consensus.cc:2229] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:05.791951 31279 raft_consensus.cc:2258] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:05.793833 31279 ts_tablet_manager.cc:1905] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:05.798440 31281 ts_tablet_manager.cc:1918] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:05.798611 31281 log.cc:1196] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/c07a18b1450347ada07b533cbc9a31b4
Aug 07 22:55:05 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.193@KRBTEST.COM
I20260807 22:55:05.799168 31281 ts_tablet_manager.cc:1939] T c07a18b1450347ada07b533cbc9a31b4 P 87eb97726e35412b95ecf9d7e204a113: Deleting consensus metadata
I20260807 22:55:05.801046 30706 catalog_manager.cc:4841] TS 87eb97726e35412b95ecf9d7e204a113 (127.9.194.193:41165): tablet c07a18b1450347ada07b533cbc9a31b4 (table test-table [id=4115dcab5a9e4853a583f1544093f9da]) successfully deleted
I20260807 22:55:05.801438 31279 ts_tablet_manager.cc:1918] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:05.801616 31279 log.cc:1196] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/c07a18b1450347ada07b533cbc9a31b4
I20260807 22:55:05.802064 31279 ts_tablet_manager.cc:1939] T c07a18b1450347ada07b533cbc9a31b4 P 1f0820ed2eef4e8fae55e03c4cedadd7: Deleting consensus metadata
I20260807 22:55:05.803594 30707 catalog_manager.cc:4841] TS 1f0820ed2eef4e8fae55e03c4cedadd7 (127.9.194.195:39053): tablet c07a18b1450347ada07b533cbc9a31b4 (table test-table [id=4115dcab5a9e4853a583f1544093f9da]) successfully deleted
I20260807 22:55:05.807403 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.773948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.9.194.193:41165)
0807 22:55:05.774150 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.774157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.787657 (+ 13500us) server_negotiation.cc:407] Connection header received
0807 22:55:05.788035 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.788040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.788127 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.788231 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:05.790112 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.791306 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.793498 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.794251 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.794366 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.800391 (+ 6025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:05.800405 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:05.800429 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:05.800478 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:05.804559 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.805247 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.805256 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.805259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.805469 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.805832 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.805837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.805841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.806290 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:05.806607 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.806771 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.807009 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:55:05.808139 30929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:05.908550 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30791
I20260807 22:55:05.929487 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30941
Aug 07 22:55:05 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.194@KRBTEST.COM
I20260807 22:55:05.944958 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:05.930391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.9.194.194:38943)
0807 22:55:05.930602 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:05.930608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:05.930628 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:55:05.930772 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:05.930778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:05.930851 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:05.930987 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:05.932371 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.933164 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.934406 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:05.934819 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:05.934965 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:05.939298 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:05.939312 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:05.939329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:05.939364 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:05.942109 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.942784 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.942800 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.942803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.942932 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:05.943307 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:05.943313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:05.943316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:05.943771 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:05.944082 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:05.944392 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 22:55:05.944610 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:55:05.945667 31075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:06.036222 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30941
I20260807 22:55:06.055646 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31088
Aug 07 22:55:06 dist-test-slave-5k54 krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.195@KRBTEST.COM
I20260807 22:55:06.071309 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:06.056468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.9.194.195:39053)
0807 22:55:06.056650 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:06.056659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:06.056683 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:55:06.056906 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:06.056911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:06.056976 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:06.057083 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:06.058380 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:06.059121 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:06.060523 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:06.060925 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:06.061060 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:06.065723 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:06.065738 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:06.065755 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:06.065787 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:06.068560 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:06.069278 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:06.069285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:06.069287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:06.069417 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:06.069819 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:06.069823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:06.069825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:06.070170 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:06.070497 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:06.070751 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 22:55:06.070987 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:55:06.071993 31224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:06.161948 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31088
I20260807 22:55:06.180691 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 30685
I20260807 22:55:06.181908 30758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:06.296497 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 30685
2026-08-07T22:55:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4428 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T22:55:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:06Z Disabled control of system clock
I20260807 22:55:06.387941 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40941
--webserver_interface=127.9.194.254
--webserver_port=0
--builtin_ntp_servers=127.9.194.212:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:55:06.728072 31290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:06.728368 31290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:06.728451 31290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:06.735669 31290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:06.735776 31290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:06.735822 31290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:06.735880 31290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:06.735924 31290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:06.745164 31290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:40941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:06.746408 31290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:06.748200 31290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:06.760006 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:06.759969 31301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:06.760064 31298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:06.760835 31290 server_base.cc:1034] running on GCE node
I20260807 22:55:06.761541 31290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:06.762867 31290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:06.764061 31290 hybrid_clock.cc:648] HybridClock initialized: now 1786143306764004 us; error 57 us; skew 500 ppm
I20260807 22:55:06.764448 31290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:06.766778 31290 webserver.cc:460] Webserver started at http://127.9.194.254:38661/ using document root <none> and password file <none>
I20260807 22:55:06.767563 31290 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:06.767684 31290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:06.768040 31290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:06.771024 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "3c904f0d8ac7491288f99ef0920d6b59"
format_stamp: "Formatted at 2026-08-07 22:55:06 on dist-test-slave-5k54"
server_key: "d251c62c2f9e4a1f29eedf538396cb17"
server_key_iv: "84ea5efbfd8e471886760fa712cddf6e"
server_key_version: "encryptionkey@0"
I20260807 22:55:06.771942 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "3c904f0d8ac7491288f99ef0920d6b59"
format_stamp: "Formatted at 2026-08-07 22:55:06 on dist-test-slave-5k54"
server_key: "d251c62c2f9e4a1f29eedf538396cb17"
server_key_iv: "84ea5efbfd8e471886760fa712cddf6e"
server_key_version: "encryptionkey@0"
I20260807 22:55:06.777381 31290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 22:55:06.781133 31307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:06.782557 31290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:55:06.782747 31290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "3c904f0d8ac7491288f99ef0920d6b59"
format_stamp: "Formatted at 2026-08-07 22:55:06 on dist-test-slave-5k54"
server_key: "d251c62c2f9e4a1f29eedf538396cb17"
server_key_iv: "84ea5efbfd8e471886760fa712cddf6e"
server_key_version: "encryptionkey@0"
I20260807 22:55:06.782992 31290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:06.809698 31290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:06.810568 31290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:06.810868 31290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:06.830018 31290 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:40941
I20260807 22:55:06.830057 31363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:40941 every 8 connection(s)
I20260807 22:55:06.831398 31290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:06.835322 31364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:06.835654 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31290
I20260807 22:55:06.835842 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:06.836447 9995 external_mini_cluster.cc:1428] Setting key f87bec0605b4603503c4f579a9bce13d
I20260807 22:55:06.848773 31364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: Bootstrap starting.
I20260807 22:55:06.850839 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:06.838534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48692 (local address 127.9.194.254:40941)
0807 22:55:06.839655 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:06.839670 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:06.839713 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:55:06.841911 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:06.841934 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:06.842495 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:06.842919 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:06.843495 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:06.844751 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:06.846439 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:06.847121 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:06.847320 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:06.848569 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:06.848577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:06.848590 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:06.849169 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:06.849249 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:06.849274 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:06.849409 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:06.849609 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 22:55:06.849911 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":405,"threads_started":1}
I20260807 22:55:06.852131 31364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:06.853253 31364 log.cc:824] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:06.856086 31364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: No bootstrap required, opened a new log
I20260807 22:55:06.863396 31364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } }
I20260807 22:55:06.863821 31364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:06.863897 31364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c904f0d8ac7491288f99ef0920d6b59, State: Initialized, Role: FOLLOWER
I20260807 22:55:06.864465 31364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } }
I20260807 22:55:06.864678 31364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:06.864776 31364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:06.864926 31364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:06.867367 31364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } }
I20260807 22:55:06.867892 31364 leader_election.cc:302] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c904f0d8ac7491288f99ef0920d6b59; no voters:
I20260807 22:55:06.868502 31364 leader_election.cc:288] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:06.868659 31369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:06.870189 31369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [term 1 LEADER]: Becoming Leader. State: Replica: 3c904f0d8ac7491288f99ef0920d6b59, State: Running, Role: LEADER
I20260807 22:55:06.870975 31364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:06.870822 31369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } }
I20260807 22:55:06.874899 31370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c904f0d8ac7491288f99ef0920d6b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } } }
I20260807 22:55:06.875345 31370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:06.876919 31379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:06.876816 31371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c904f0d8ac7491288f99ef0920d6b59. Latest consensus state: current_term: 1 leader_uuid: "3c904f0d8ac7491288f99ef0920d6b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c904f0d8ac7491288f99ef0920d6b59" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40941 } } }
I20260807 22:55:06.877579 31371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:06.884908 31379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:06.897099 31379 catalog_manager.cc:1347] Generated new cluster ID: 75f06e9ece9b4143a9a53e95b4d39355
I20260807 22:55:06.897220 31379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:06.907637 31379 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:06.908824 31379 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:06.916395 31379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: Generated new TSK 0
I20260807 22:55:06.917201 31379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:06.924817 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.193:0
--local_ip_for_outbound_sockets=127.9.194.193
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--builtin_ntp_servers=127.9.194.212:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:55:07.272037 31391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:07.272434 31391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:07.272639 31391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:07.280087 31391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:07.280189 31391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:07.280357 31391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.193
I20260807 22:55:07.289990 31391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.194.193
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.193
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:07.291314 31391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:07.293172 31391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:07.306443 31398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:07.306404 31401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:07.306357 31399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:07.307075 31391 server_base.cc:1034] running on GCE node
I20260807 22:55:07.307787 31391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:07.308730 31391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:07.309973 31391 hybrid_clock.cc:648] HybridClock initialized: now 1786143307309932 us; error 56 us; skew 500 ppm
I20260807 22:55:07.310293 31391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:07.317353 31391 webserver.cc:460] Webserver started at http://127.9.194.193:35791/ using document root <none> and password file <none>
I20260807 22:55:07.318089 31391 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:07.318223 31391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:07.318552 31391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:07.321444 31391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data/instance:
uuid: "f507d927614644c8b5490c71a14ce7ec"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "ae0e678a90d2fed32b49d227f919a751"
server_key_iv: "50237adde389f3bfd7bc9075520be4e2"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.322234 31391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance:
uuid: "f507d927614644c8b5490c71a14ce7ec"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "ae0e678a90d2fed32b49d227f919a751"
server_key_iv: "50237adde389f3bfd7bc9075520be4e2"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.327216 31391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:55:07.330753 31408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.332247 31391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:07.332436 31391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal
uuid: "f507d927614644c8b5490c71a14ce7ec"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "ae0e678a90d2fed32b49d227f919a751"
server_key_iv: "50237adde389f3bfd7bc9075520be4e2"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.332660 31391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:07.350076 31391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:07.350881 31391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:07.351197 31391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:07.352267 31391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:07.353680 31391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:07.353812 31391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.353936 31391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:07.354033 31391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.387698 31391 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.193:33495
I20260807 22:55:07.387787 31530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.193:33495 every 8 connection(s)
I20260807 22:55:07.389042 31391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/data/info.pb
I20260807 22:55:07.396004 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31391
I20260807 22:55:07.396221 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal/instance
I20260807 22:55:07.396656 9995 external_mini_cluster.cc:1428] Setting key 84244da0baf8d4f90163f80dd3338d7b
I20260807 22:55:07.399279 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.194:0
--local_ip_for_outbound_sockets=127.9.194.194
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--builtin_ntp_servers=127.9.194.212:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:55:07.402027 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:07.391724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.193:59265 (local address 127.9.194.254:40941)
0807 22:55:07.392181 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:07.392185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:07.393417 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 22:55:07.395220 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:07.395226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:07.395317 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:07.395450 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:07.396669 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.397411 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.398957 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.399372 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.399549 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:07.401180 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:07.401184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:07.401187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:07.401204 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:07.401243 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:07.401254 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:07.401319 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:07.401580 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 22:55:07.401651 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":253,"threads_started":1}
I20260807 22:55:07.403510 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:07.392396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:40941 (local address 127.9.194.193:59265)
0807 22:55:07.393259 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:07.393302 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:07.394960 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:07.395726 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:07.395735 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:07.395819 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:07.396496 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:07.396515 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.397583 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.397588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:07.398804 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:07.398816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.399693 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.399702 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:07.400033 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:07.400802 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:07.400808 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:07.400865 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:07.400976 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:07.401349 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:07.401362 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:07.401381 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 22:55:07.401569 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 22:55:07.401981 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":247,"threads_started":1}
I20260807 22:55:07.405030 31531 heartbeater.cc:347] Connected to a master server at 127.9.194.254:40941
I20260807 22:55:07.405331 31531 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:07.406246 31531 heartbeater.cc:511] Master 127.9.194.254:40941 requested a full tablet report, sending...
I20260807 22:55:07.408541 31326 ts_manager.cc:194] Registered new tserver with Master: f507d927614644c8b5490c71a14ce7ec (127.9.194.193:33495)
I20260807 22:55:07.410778 31326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.193:59265
W20260807 22:55:07.736301 31536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:07.736584 31536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:07.736745 31536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:07.744225 31536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:07.744345 31536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:07.744488 31536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.194
I20260807 22:55:07.753649 31536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.194.194
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.194
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:07.754926 31536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:07.756707 31536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:07.770037 31541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:07.770107 31542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:07.770118 31544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:07.776332 31536 server_base.cc:1034] running on GCE node
I20260807 22:55:07.776983 31536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:07.777961 31536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:07.779204 31536 hybrid_clock.cc:648] HybridClock initialized: now 1786143307779166 us; error 54 us; skew 500 ppm
I20260807 22:55:07.779593 31536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:07.785986 31536 webserver.cc:460] Webserver started at http://127.9.194.194:38499/ using document root <none> and password file <none>
I20260807 22:55:07.786706 31536 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:07.786839 31536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:07.787173 31536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:07.790176 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data/instance:
uuid: "6861e600a08b4ed193cbad6101196018"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "1a994a307aa0bc683c9a49e228f13253"
server_key_iv: "4abd70a99d8a213f9d2b5945da27371c"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.790967 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance:
uuid: "6861e600a08b4ed193cbad6101196018"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "1a994a307aa0bc683c9a49e228f13253"
server_key_iv: "4abd70a99d8a213f9d2b5945da27371c"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.796154 31536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 22:55:07.799652 31551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.800925 31536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:07.801105 31536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal
uuid: "6861e600a08b4ed193cbad6101196018"
format_stamp: "Formatted at 2026-08-07 22:55:07 on dist-test-slave-5k54"
server_key: "1a994a307aa0bc683c9a49e228f13253"
server_key_iv: "4abd70a99d8a213f9d2b5945da27371c"
server_key_version: "encryptionkey@0"
I20260807 22:55:07.801338 31536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:07.817224 31536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:07.818053 31536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:07.818368 31536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:07.819339 31536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:07.820815 31536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:07.820946 31536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.821089 31536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:07.821177 31536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:07.857518 31536 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.194:43129
I20260807 22:55:07.857573 31674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.194:43129 every 8 connection(s)
I20260807 22:55:07.858883 31536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/data/info.pb
I20260807 22:55:07.862265 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31536
I20260807 22:55:07.862490 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal/instance
I20260807 22:55:07.863092 9995 external_mini_cluster.cc:1428] Setting key 30b3601a508a964216b063c802db1879
I20260807 22:55:07.866103 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.194.195:0
--local_ip_for_outbound_sockets=127.9.194.195
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--builtin_ntp_servers=127.9.194.212:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:55:07.872503 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:07.861551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:45739 (local address 127.9.194.254:40941)
0807 22:55:07.861745 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:07.861750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:07.863484 (+ 1734us) server_negotiation.cc:407] Connection header received
0807 22:55:07.865856 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:07.865861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:07.865942 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:07.866059 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:07.867154 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.867924 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.869615 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.870016 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.870157 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:07.871592 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:07.871599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:07.871603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:07.871630 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:07.871664 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:07.871676 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:07.871771 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:07.872023 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 22:55:07.872126 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:55:07.874449 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:07.862241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:40941 (local address 127.9.194.194:45739)
0807 22:55:07.863302 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:07.863340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:07.865597 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:07.866308 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:07.866320 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:07.866441 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:07.866991 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:07.867009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.868083 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.868088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:07.869466 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:07.869477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:07.870233 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:07.870239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:07.870464 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:07.871212 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:07.871215 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:07.871264 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:07.871379 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:07.871769 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:07.871782 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:07.871792 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:55:07.872002 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 22:55:07.873144 (+ 1142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":342,"threads_started":1}
I20260807 22:55:07.876233 31675 heartbeater.cc:347] Connected to a master server at 127.9.194.254:40941
I20260807 22:55:07.876633 31675 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:07.877660 31675 heartbeater.cc:511] Master 127.9.194.254:40941 requested a full tablet report, sending...
I20260807 22:55:07.880910 31326 ts_manager.cc:194] Registered new tserver with Master: 6861e600a08b4ed193cbad6101196018 (127.9.194.194:43129)
I20260807 22:55:07.882179 31326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.194:45739
W20260807 22:55:08.214404 31679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:08.214745 31679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:08.214921 31679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:08.224305 31679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:08.224464 31679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:08.224653 31679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.194.195
I20260807 22:55:08.235014 31679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.194.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.194.195
--webserver_port=0
--tserver_master_addrs=127.9.194.254:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.194.195
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:08.236542 31679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:08.238457 31679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:08.251958 31684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:08.251987 31685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:08.251852 31687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:08.253072 31679 server_base.cc:1034] running on GCE node
I20260807 22:55:08.253710 31679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:08.254657 31679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:08.255908 31679 hybrid_clock.cc:648] HybridClock initialized: now 1786143308255838 us; error 62 us; skew 500 ppm
I20260807 22:55:08.256254 31679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:55:08.258711 31679 webserver.cc:460] Webserver started at http://127.9.194.195:45693/ using document root <none> and password file <none>
I20260807 22:55:08.259480 31679 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:08.259624 31679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:08.259958 31679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:08.262799 31679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data/instance:
uuid: "40bfe184b5a442258a6436af02e2d15e"
format_stamp: "Formatted at 2026-08-07 22:55:08 on dist-test-slave-5k54"
server_key: "4b4961e2c8bf8cbb164e350565a12737"
server_key_iv: "a9da655058377e83edace974d47da55f"
server_key_version: "encryptionkey@0"
I20260807 22:55:08.263651 31679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance:
uuid: "40bfe184b5a442258a6436af02e2d15e"
format_stamp: "Formatted at 2026-08-07 22:55:08 on dist-test-slave-5k54"
server_key: "4b4961e2c8bf8cbb164e350565a12737"
server_key_iv: "a9da655058377e83edace974d47da55f"
server_key_version: "encryptionkey@0"
I20260807 22:55:08.268699 31679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 22:55:08.272317 31693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:08.273857 31679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:08.274045 31679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal
uuid: "40bfe184b5a442258a6436af02e2d15e"
format_stamp: "Formatted at 2026-08-07 22:55:08 on dist-test-slave-5k54"
server_key: "4b4961e2c8bf8cbb164e350565a12737"
server_key_iv: "a9da655058377e83edace974d47da55f"
server_key_version: "encryptionkey@0"
I20260807 22:55:08.274322 31679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:08.299747 31679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:08.300606 31679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:08.300997 31679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:08.302114 31679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:55:08.303637 31679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:55:08.303730 31679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:08.303809 31679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:55:08.303879 31679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:08.337745 31679 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.195:33421
I20260807 22:55:08.337827 31816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.195:33421 every 8 connection(s)
I20260807 22:55:08.339078 31679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/data/info.pb
I20260807 22:55:08.342875 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31679
I20260807 22:55:08.343082 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal/instance
I20260807 22:55:08.343667 9995 external_mini_cluster.cc:1428] Setting key 61634bc8e295a6913c641f2f4f8b0d1d
I20260807 22:55:08.352348 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.341759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.195:50759 (local address 127.9.194.254:40941)
0807 22:55:08.341984 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.341990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.343587 (+ 1597us) server_negotiation.cc:407] Connection header received
0807 22:55:08.345421 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.345427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.345513 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.345614 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:08.347122 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.348082 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.349602 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.350058 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.350164 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.351644 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:08.351649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:08.351653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:08.351672 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:08.351706 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:08.351719 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:08.351799 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.352021 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.352103 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:55:08.353283 31819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.342356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:40941 (local address 127.9.194.195:50759)
0807 22:55:08.343401 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.343468 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.345172 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.346223 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.346234 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.346364 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:08.346942 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.346959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.348229 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.348233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.349447 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.349460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.350245 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.350250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.350455 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.351173 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:55:08.351175 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:08.351213 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:55:08.351291 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:08.351810 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:08.351817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:08.351823 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.351964 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.352247 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":327,"threads_started":1}
I20260807 22:55:08.354730 31817 heartbeater.cc:347] Connected to a master server at 127.9.194.254:40941
I20260807 22:55:08.355022 31817 heartbeater.cc:464] Registering TS with master...
I20260807 22:55:08.355779 31817 heartbeater.cc:511] Master 127.9.194.254:40941 requested a full tablet report, sending...
I20260807 22:55:08.357374 31326 ts_manager.cc:194] Registered new tserver with Master: 40bfe184b5a442258a6436af02e2d15e (127.9.194.195:33421)
I20260807 22:55:08.358388 31326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.194.195:50759
I20260807 22:55:08.359221 9995 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:55:08.369938 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.363380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.9.194.254:40941)
0807 22:55:08.363707 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.363716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.364098 (+ 382us) server_negotiation.cc:407] Connection header received
0807 22:55:08.364315 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.364328 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.364412 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.364529 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:08.365174 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.365916 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.367379 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.367859 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.368026 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.369109 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:08.369115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:08.369118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:08.369136 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:08.369168 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:08.369181 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:08.369286 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.369479 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.369596 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 22:55:08.374308 31325 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:55:08.381382 31325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:55:08.404140 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.395216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33421 (local address 127.0.0.1:46534)
0807 22:55:08.395770 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.395787 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.395911 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.397604 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.397609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.397634 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:08.398006 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.398022 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.399976 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.399981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.401682 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.401692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.403113 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.403119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.403343 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.403402 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.403552 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.403761 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":27,"thread_start_us":275,"threads_started":1}
I20260807 22:55:08.404464 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.393951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.9.194.194:43129)
0807 22:55:08.394684 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.394693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.394910 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:55:08.395178 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.395186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.395500 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.395698 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:08.396763 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.398460 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.401549 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.402998 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.403129 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.403381 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.403939 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.404127 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":465,"threads_started":1}
I20260807 22:55:08.404852 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.393937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.194:43129 (local address 127.0.0.1:35254)
0807 22:55:08.394781 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.394811 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.394964 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.395972 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.395980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.396012 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:08.396540 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.396558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.398598 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.398605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.401261 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.401282 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.403537 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.403549 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.403728 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.403767 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.403859 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.403989 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":66,"thread_start_us":483,"threads_started":1}
I20260807 22:55:08.405378 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.395598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.9.194.195:33421)
0807 22:55:08.397100 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.397110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.397144 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:55:08.397236 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.397242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.397429 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.397614 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:08.398237 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.399803 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.401868 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.402883 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.404449 (+ 1566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.404804 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.404886 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.405074 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1333,"thread_start_us":1485,"threads_started":1}
I20260807 22:55:08.406493 31836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.395836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:33495 (local address 127.0.0.1:37588)
0807 22:55:08.396459 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.396480 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.396593 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.398865 (+ 2272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.398872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.398905 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:08.399366 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.399379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.401255 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.401261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.403485 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.403499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.405743 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.405750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.405943 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.405975 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.406082 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.406210 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":709,"threads_started":1}
I20260807 22:55:08.409096 31745 tablet_service.cc:1467] Processing CreateTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe (DEFAULT_TABLE table=test-table [id=967fd5e890c04797a064de97e1ff6289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:08.409096 31602 tablet_service.cc:1467] Processing CreateTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe (DEFAULT_TABLE table=test-table [id=967fd5e890c04797a064de97e1ff6289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:08.410329 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.396367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.9.194.193:33495)
0807 22:55:08.398302 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.398309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.398340 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:55:08.398445 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.398455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.398671 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.398865 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:08.399584 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.401128 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.403711 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.405560 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.405686 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.405962 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.409930 (+ 3968us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.410071 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1630,"threads_started":1}
I20260807 22:55:08.410655 31745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab8974f43974b5fafe7b7a93e52dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:08.411506 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab8974f43974b5fafe7b7a93e52dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:08.414032 31531 heartbeater.cc:503] Master 127.9.194.254:40941 was elected leader, sending a full tablet report...
I20260807 22:55:08.413919 31458 tablet_service.cc:1467] Processing CreateTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe (DEFAULT_TABLE table=test-table [id=967fd5e890c04797a064de97e1ff6289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:55:08.415483 31458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab8974f43974b5fafe7b7a93e52dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:08.427188 31840 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Bootstrap starting.
I20260807 22:55:08.432243 31838 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Bootstrap starting.
I20260807 22:55:08.432459 31840 tablet_bootstrap.cc:654] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:08.434303 31840 log.cc:824] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:08.435724 31838 tablet_bootstrap.cc:654] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:08.437036 31838 log.cc:824] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:08.436686 31839 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Bootstrap starting.
I20260807 22:55:08.438954 31840 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: No bootstrap required, opened a new log
I20260807 22:55:08.439232 31840 ts_tablet_manager.cc:1397] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 22:55:08.440214 31838 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: No bootstrap required, opened a new log
I20260807 22:55:08.440485 31838 ts_tablet_manager.cc:1397] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260807 22:55:08.441279 31839 tablet_bootstrap.cc:654] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:08.442525 31839 log.cc:824] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:08.446421 31839 tablet_bootstrap.cc:492] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: No bootstrap required, opened a new log
I20260807 22:55:08.446789 31839 ts_tablet_manager.cc:1397] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 22:55:08.448707 31840 raft_consensus.cc:348] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.449215 31840 raft_consensus.cc:374] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:08.449322 31840 raft_consensus.cc:729] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6861e600a08b4ed193cbad6101196018, State: Initialized, Role: FOLLOWER
I20260807 22:55:08.449698 31838 raft_consensus.cc:348] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.450095 31838 raft_consensus.cc:374] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:08.449999 31840 consensus_queue.cc:260] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.450219 31838 raft_consensus.cc:729] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40bfe184b5a442258a6436af02e2d15e, State: Initialized, Role: FOLLOWER
I20260807 22:55:08.450824 31838 consensus_queue.cc:260] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.452005 31817 heartbeater.cc:503] Master 127.9.194.254:40941 was elected leader, sending a full tablet report...
I20260807 22:55:08.452569 31838 ts_tablet_manager.cc:1428] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260807 22:55:08.456403 31675 heartbeater.cc:503] Master 127.9.194.254:40941 was elected leader, sending a full tablet report...
I20260807 22:55:08.457681 31840 ts_tablet_manager.cc:1428] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Time spent starting tablet: real 0.018s user 0.007s sys 0.010s
I20260807 22:55:08.458451 31839 raft_consensus.cc:348] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.458925 31839 raft_consensus.cc:374] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:08.459040 31839 raft_consensus.cc:729] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f507d927614644c8b5490c71a14ce7ec, State: Initialized, Role: FOLLOWER
I20260807 22:55:08.459700 31839 consensus_queue.cc:260] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.461441 31839 ts_tablet_manager.cc:1428] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 22:55:08.489378 31844 raft_consensus.cc:482] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:55:08.489790 31844 raft_consensus.cc:504] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.491379 31844 leader_election.cc:288] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40bfe184b5a442258a6436af02e2d15e (127.9.194.195:33421), f507d927614644c8b5490c71a14ce7ec (127.9.194.193:33495)
I20260807 22:55:08.500033 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.492854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:36765 (local address 127.9.194.193:33495)
0807 22:55:08.493043 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.493049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.493395 (+ 346us) server_negotiation.cc:407] Connection header received
0807 22:55:08.493727 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.493733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.493827 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.493940 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:08.494575 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.495261 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.497682 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.498942 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.499091 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.499159 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.499645 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.499792 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:55:08.500061 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.491872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.194.194:52153 (local address 127.9.194.195:33421)
0807 22:55:08.492053 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.492058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.492734 (+ 676us) server_negotiation.cc:407] Connection header received
0807 22:55:08.492973 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.492980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.493076 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.493141 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:55:08.493934 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.494952 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.497611 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.498926 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.499091 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.499159 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.499645 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.499792 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:55:08.500211 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.492627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.193:33495 (local address 127.9.194.194:36765)
0807 22:55:08.493236 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.493250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.493491 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.493972 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.493977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.494017 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:08.494358 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.494369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.495401 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.495406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.497476 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.497498 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.499195 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.499201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.499408 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.499498 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.499589 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.499780 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":65,"spinlock_wait_cycles":25984,"thread_start_us":344,"threads_started":1}
I20260807 22:55:08.500239 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.491904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.195:33421 (local address 127.9.194.194:52153)
0807 22:55:08.492596 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:08.492617 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:08.492781 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:08.493299 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:08.493303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:08.493324 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:55:08.493799 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.493810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.495081 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.495088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.497388 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:08.497403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.499186 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.499193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:08.499416 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.499492 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 22:55:08.499589 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:55:08.499780 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":101,"thread_start_us":343,"threads_started":1}
I20260807 22:55:08.501250 31768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab8974f43974b5fafe7b7a93e52dcbe" candidate_uuid: "6861e600a08b4ed193cbad6101196018" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bfe184b5a442258a6436af02e2d15e" is_pre_election: true
I20260807 22:55:08.501477 31480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab8974f43974b5fafe7b7a93e52dcbe" candidate_uuid: "6861e600a08b4ed193cbad6101196018" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f507d927614644c8b5490c71a14ce7ec" is_pre_election: true
I20260807 22:55:08.501890 31768 raft_consensus.cc:2454] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6861e600a08b4ed193cbad6101196018 in term 0.
I20260807 22:55:08.502034 31480 raft_consensus.cc:2454] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6861e600a08b4ed193cbad6101196018 in term 0.
I20260807 22:55:08.502553 31554 leader_election.cc:302] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40bfe184b5a442258a6436af02e2d15e, 6861e600a08b4ed193cbad6101196018; no voters:
I20260807 22:55:08.502954 31844 raft_consensus.cc:2781] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:55:08.503104 31844 raft_consensus.cc:482] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:55:08.503232 31844 raft_consensus.cc:3037] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:08.505784 31844 raft_consensus.cc:504] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.506446 31844 leader_election.cc:288] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [CANDIDATE]: Term 1 election: Requested vote from peers 40bfe184b5a442258a6436af02e2d15e (127.9.194.195:33421), f507d927614644c8b5490c71a14ce7ec (127.9.194.193:33495)
I20260807 22:55:08.507004 31480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab8974f43974b5fafe7b7a93e52dcbe" candidate_uuid: "6861e600a08b4ed193cbad6101196018" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f507d927614644c8b5490c71a14ce7ec"
I20260807 22:55:08.507077 31768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab8974f43974b5fafe7b7a93e52dcbe" candidate_uuid: "6861e600a08b4ed193cbad6101196018" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bfe184b5a442258a6436af02e2d15e"
I20260807 22:55:08.507230 31480 raft_consensus.cc:3037] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:08.507263 31768 raft_consensus.cc:3037] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:08.510551 31480 raft_consensus.cc:2454] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6861e600a08b4ed193cbad6101196018 in term 1.
I20260807 22:55:08.510551 31768 raft_consensus.cc:2454] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6861e600a08b4ed193cbad6101196018 in term 1.
I20260807 22:55:08.510952 31554 leader_election.cc:302] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6861e600a08b4ed193cbad6101196018, f507d927614644c8b5490c71a14ce7ec; no voters:
I20260807 22:55:08.511225 31844 raft_consensus.cc:2781] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:08.511693 31844 raft_consensus.cc:686] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 1 LEADER]: Becoming Leader. State: Replica: 6861e600a08b4ed193cbad6101196018, State: Running, Role: LEADER
I20260807 22:55:08.512174 31844 consensus_queue.cc:237] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } }
I20260807 22:55:08.517270 31326 catalog_manager.cc:5495] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6861e600a08b4ed193cbad6101196018 (127.9.194.194). New cstate: current_term: 1 leader_uuid: "6861e600a08b4ed193cbad6101196018" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6861e600a08b4ed193cbad6101196018" member_type: VOTER last_known_addr { host: "127.9.194.194" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260807 22:55:08.561110 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.555231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.9.194.194:43129)
0807 22:55:08.555472 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.555480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.555628 (+ 148us) server_negotiation.cc:407] Connection header received
0807 22:55:08.555922 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.555927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.556033 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.556176 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:08.556573 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.557305 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.558991 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.559402 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.559561 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.560025 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:08.560529 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:08.560642 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.560771 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.560833 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:55:08.569947 31768 raft_consensus.cc:1261] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 1 FOLLOWER]: Refusing update from remote peer 6861e600a08b4ed193cbad6101196018: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:08.569919 31480 raft_consensus.cc:1261] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 1 FOLLOWER]: Refusing update from remote peer 6861e600a08b4ed193cbad6101196018: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:08.570963 31851 consensus_queue.cc:1035] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40bfe184b5a442258a6436af02e2d15e" member_type: VOTER last_known_addr { host: "127.9.194.195" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:08.571386 31844 consensus_queue.cc:1035] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f507d927614644c8b5490c71a14ce7ec" member_type: VOTER last_known_addr { host: "127.9.194.193" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:08.598631 31326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:08.598979 31326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:55:08.602900 31326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c904f0d8ac7491288f99ef0920d6b59: Sending DeleteTablet for 3 replicas of tablet dab8974f43974b5fafe7b7a93e52dcbe
I20260807 22:55:08.604127 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:08 UTC) from {username='slave'} at 127.0.0.1:35254
I20260807 22:55:08.604667 31745 tablet_service.cc:1514] Processing DeleteTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:08 UTC) from {username='slave'} at 127.0.0.1:46534
I20260807 22:55:08.605197 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31391
I20260807 22:55:08.605506 31458 tablet_service.cc:1514] Processing DeleteTablet for tablet dab8974f43974b5fafe7b7a93e52dcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:55:08 UTC) from {username='slave'} at 127.0.0.1:37588
I20260807 22:55:08.615186 31865 tablet_replica.cc:330] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: stopping tablet replica
I20260807 22:55:08.615779 31865 raft_consensus.cc:2229] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:55:08.616710 31865 raft_consensus.cc:2258] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:08.617897 31867 tablet_replica.cc:330] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: stopping tablet replica
I20260807 22:55:08.618336 31867 raft_consensus.cc:2229] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:08.618837 31867 raft_consensus.cc:2258] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:08.619410 31865 ts_tablet_manager.cc:1905] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:08.622723 31867 ts_tablet_manager.cc:1905] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:08.627566 31866 tablet_replica.cc:330] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: stopping tablet replica
I20260807 22:55:08.628158 31866 raft_consensus.cc:2229] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:55:08.628614 31865 ts_tablet_manager.cc:1918] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:08.628800 31865 log.cc:1196] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-1/wal/wals/dab8974f43974b5fafe7b7a93e52dcbe
I20260807 22:55:08.629444 31865 ts_tablet_manager.cc:1939] T dab8974f43974b5fafe7b7a93e52dcbe P 6861e600a08b4ed193cbad6101196018: Deleting consensus metadata
I20260807 22:55:08.629887 31867 ts_tablet_manager.cc:1918] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:08.630100 31867 log.cc:1196] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-0/wal/wals/dab8974f43974b5fafe7b7a93e52dcbe
I20260807 22:55:08.630635 31867 ts_tablet_manager.cc:1939] T dab8974f43974b5fafe7b7a93e52dcbe P f507d927614644c8b5490c71a14ce7ec: Deleting consensus metadata
I20260807 22:55:08.630620 31866 raft_consensus.cc:2258] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:55:08.631567 31310 catalog_manager.cc:4841] TS 6861e600a08b4ed193cbad6101196018 (127.9.194.194:43129): tablet dab8974f43974b5fafe7b7a93e52dcbe (table test-table [id=967fd5e890c04797a064de97e1ff6289]) successfully deleted
I20260807 22:55:08.632771 31310 catalog_manager.cc:4841] TS f507d927614644c8b5490c71a14ce7ec (127.9.194.193:33495): tablet dab8974f43974b5fafe7b7a93e52dcbe (table test-table [id=967fd5e890c04797a064de97e1ff6289]) successfully deleted
I20260807 22:55:08.633991 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.606536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37598 (local address 127.9.194.193:33495)
0807 22:55:08.606707 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.606713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.620496 (+ 13783us) server_negotiation.cc:407] Connection header received
0807 22:55:08.621060 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.621067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.621181 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.621293 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:08.622445 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.623790 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.628441 (+ 4651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.629251 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.629387 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.632225 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:08.632239 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:08.632259 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:08.633081 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:08.633147 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:08.633173 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:08.633310 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.633490 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.633614 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:55:08.633987 31866 ts_tablet_manager.cc:1905] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:55:08.634763 31526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:08.642650 31866 ts_tablet_manager.cc:1918] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:55:08.642935 31866 log.cc:1196] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Deleting WAL directory at /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142983724342-9995-0/minicluster-data/ts-2/wal/wals/dab8974f43974b5fafe7b7a93e52dcbe
I20260807 22:55:08.643602 31866 ts_tablet_manager.cc:1939] T dab8974f43974b5fafe7b7a93e52dcbe P 40bfe184b5a442258a6436af02e2d15e: Deleting consensus metadata
I20260807 22:55:08.645689 31310 catalog_manager.cc:4841] TS 40bfe184b5a442258a6436af02e2d15e (127.9.194.195:33421): tablet dab8974f43974b5fafe7b7a93e52dcbe (table test-table [id=967fd5e890c04797a064de97e1ff6289]) successfully deleted
I20260807 22:55:08.748770 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31391
I20260807 22:55:08.777619 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31536
I20260807 22:55:08.787374 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.778615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.9.194.194:43129)
0807 22:55:08.778822 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.778830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.778966 (+ 136us) server_negotiation.cc:407] Connection header received
0807 22:55:08.779135 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.779141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.779249 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.779364 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:08.779875 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.781119 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.783020 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.783989 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.784141 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.785386 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:08.785400 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:08.785422 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:08.786361 (+ 939us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:08.786428 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:08.786457 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:08.786574 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.786738 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.786858 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:55:08.788235 31669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:08.932440 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31536
I20260807 22:55:08.955099 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31679
I20260807 22:55:08.961863 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:08.955964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46540 (local address 127.9.194.195:33421)
0807 22:55:08.956121 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:08.956126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:08.956172 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:55:08.956345 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:08.956350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:08.956457 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:08.956541 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:08.956919 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.957655 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.959067 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:08.959502 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:08.959603 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:08.960583 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:08.960592 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:08.960606 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:08.961150 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:08.961197 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:08.961215 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:08.961300 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:08.961415 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 22:55:08.961500 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:55:08.962525 31811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:09.052000 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31679
I20260807 22:55:09.070755 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31290
I20260807 22:55:09.071954 31359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:09.181069 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31290
2026-08-07T22:55:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2887 ms)
[----------] 6 tests from EncryptionPolicyTest (22755 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:09 dist-test-slave-5k54 krb5kdc[31876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:09 dist-test-slave-5k54 krb5kdc[31876](info): set up 2 sockets
Aug 07 22:55:09 dist-test-slave-5k54 krb5kdc[31876](info): commencing operation
krb5kdc: starting...
W20260807 22:55:10.356751 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:10 dist-test-slave-5k54 krb5kdc[31876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:10.665162 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:11.006245 31891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:11.006558 31891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:11.006732 31891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:11.013986 31891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:11.014099 31891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:11.014194 31891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:11.014309 31891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:11.014390 31891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:11.025161 31891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:11.026540 31891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:11.027915 31891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:11.039928 31897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:11.040016 31896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:11.040043 31899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:11.041186 31891 server_base.cc:1034] running on GCE node
I20260807 22:55:11.041903 31891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:11.043228 31891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:11.044431 31891 hybrid_clock.cc:648] HybridClock initialized: now 1786143311044378 us; error 41 us; skew 500 ppm
I20260807 22:55:11.044735 31891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:11 dist-test-slave-5k54 krb5kdc[31876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:11.047973 31891 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:11.049527 31891 webserver.cc:460] Webserver started at http://127.0.0.1:38101/ using document root <none> and password file <none>
I20260807 22:55:11.050253 31891 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:11.050388 31891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:11.050725 31891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:11.053722 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "b37e470019074a348b5716521092f63b"
format_stamp: "Formatted at 2026-08-07 22:55:11 on dist-test-slave-5k54"
server_key: "460804be2a6fb28ba8fe7f2a8a779c09"
server_key_iv: "863ca3261cf22d4ad8216cdd72a75e9a"
server_key_version: "encryptionkey@0"
I20260807 22:55:11.054512 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "b37e470019074a348b5716521092f63b"
format_stamp: "Formatted at 2026-08-07 22:55:11 on dist-test-slave-5k54"
server_key: "460804be2a6fb28ba8fe7f2a8a779c09"
server_key_iv: "863ca3261cf22d4ad8216cdd72a75e9a"
server_key_version: "encryptionkey@0"
I20260807 22:55:11.059729 31891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 22:55:11.063146 31906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:11.064561 31891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:11.064704 31891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "b37e470019074a348b5716521092f63b"
format_stamp: "Formatted at 2026-08-07 22:55:11 on dist-test-slave-5k54"
server_key: "460804be2a6fb28ba8fe7f2a8a779c09"
server_key_iv: "863ca3261cf22d4ad8216cdd72a75e9a"
server_key_version: "encryptionkey@0"
I20260807 22:55:11.064957 31891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:11.083743 31891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:11.087816 31891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:11.088119 31891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:11.108165 31891 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39607
I20260807 22:55:11.108214 31964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39607 every 8 connection(s)
I20260807 22:55:11.109612 31891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:11.113488 31965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:11.116212 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 31891
I20260807 22:55:11.116438 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:11.117247 9995 external_mini_cluster.cc:1428] Setting key 6c222e94004598a182d45500a05db623
I20260807 22:55:11.126696 31965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b: Bootstrap starting.
I20260807 22:55:11.130674 31965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:11.132037 31965 log.cc:824] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:55:11 dist-test-slave-5k54 krb5kdc[31876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:55:11.134961 31965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b: No bootstrap required, opened a new log
I20260807 22:55:11.141297 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:11.119396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33173 (local address 127.0.0.1:39607)
0807 22:55:11.120324 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:11.120335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:11.120377 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:55:11.122032 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:11.122055 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:11.122454 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:11.122787 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:11.124551 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:11.125829 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:11.127610 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:11.128289 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:11.128437 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:11.133770 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:11.133777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:11.133791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:11.133825 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:11.137013 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:11.137841 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:11.137847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:11.137853 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:11.138028 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:11.138521 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:11.138525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:11.138527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:11.139066 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:11.139434 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:11.139988 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 22:55:11.140296 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":426,"threads_started":1}
I20260807 22:55:11.142906 31965 raft_consensus.cc:348] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } }
I20260807 22:55:11.143282 31965 raft_consensus.cc:374] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:11.143404 31965 raft_consensus.cc:729] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37e470019074a348b5716521092f63b, State: Initialized, Role: FOLLOWER
I20260807 22:55:11.144050 31965 consensus_queue.cc:260] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } }
I20260807 22:55:11.144392 31965 raft_consensus.cc:388] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:11.144562 31965 raft_consensus.cc:482] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:11.144757 31965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:11.145313 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:11.147010 31965 raft_consensus.cc:504] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } }
I20260807 22:55:11.147568 31965 leader_election.cc:302] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b37e470019074a348b5716521092f63b; no voters:
I20260807 22:55:11.148202 31965 leader_election.cc:288] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:11.148321 31971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:11.148984 31971 raft_consensus.cc:686] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [term 1 LEADER]: Becoming Leader. State: Replica: b37e470019074a348b5716521092f63b, State: Running, Role: LEADER
I20260807 22:55:11.149659 31971 consensus_queue.cc:237] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } }
I20260807 22:55:11.152944 31975 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b37e470019074a348b5716521092f63b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } } }
I20260807 22:55:11.153406 31975 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:11.157667 31975 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b37e470019074a348b5716521092f63b. Latest consensus state: current_term: 1 leader_uuid: "b37e470019074a348b5716521092f63b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e470019074a348b5716521092f63b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39607 } } }
I20260807 22:55:11.157922 31975 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:11.161042 31965 sys_catalog.cc:564] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:11.163409 31978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:11.176015 31978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:11.176470 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:11.159190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35101 (local address 127.0.0.1:39607)
0807 22:55:11.159436 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:11.159444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:11.159474 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:55:11.159678 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:11.159687 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:11.159791 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:11.159880 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:11.162349 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:11.163889 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:11.165185 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:11.165665 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:11.165806 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:11.168396 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:11.168403 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:11.168408 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:11.168450 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:11.171772 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:11.172851 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:11.172855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:11.172858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:11.173025 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:11.173667 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:11.173672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:11.173676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:11.174102 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:11.175559 (+ 1457us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:11.175754 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 22:55:11.175960 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:55:11.186719 31978 catalog_manager.cc:1347] Generated new cluster ID: a310b813bbd74370903dac07e2e508c2
I20260807 22:55:11.186854 31978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:11.204423 31978 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:11.205762 31978 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:11.219208 31978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b37e470019074a348b5716521092f63b: Generated new TSK 0
I20260807 22:55:11.220144 31978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:11.230577 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 31891
I20260807 22:55:11.232021 31959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:11.324651 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 31891
W20260807 22:55:11.345541 31874 connection.cc:535] client connection to 127.0.0.1:39607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39607): Connection reset by peer (error 104)
2026-08-07T22:55:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:11 dist-test-slave-5k54 krb5kdc[32000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:11 dist-test-slave-5k54 krb5kdc[32000](info): set up 2 sockets
Aug 07 22:55:11 dist-test-slave-5k54 krb5kdc[32000](info): commencing operation
krb5kdc: starting...
W20260807 22:55:12.492734 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:12 dist-test-slave-5k54 krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:12.810318 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:13.151813 32013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:13.152161 32013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:13.152331 32013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:13.159614 32013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:13.159762 32013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:13.159879 32013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:13.159989 32013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:13.160068 32013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:13.169945 32013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:13.171249 32013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:13.172653 32013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:13.184532 32018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:13.184484 32019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:13.184451 32021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:13.185370 32013 server_base.cc:1034] running on GCE node
I20260807 22:55:13.186162 32013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:13.187582 32013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:13.188818 32013 hybrid_clock.cc:648] HybridClock initialized: now 1786143313188760 us; error 44 us; skew 500 ppm
I20260807 22:55:13.189199 32013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:13 dist-test-slave-5k54 krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:13.202311 32013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:13.203992 32013 webserver.cc:460] Webserver started at http://127.0.0.1:44945/ using document root <none> and password file <none>
I20260807 22:55:13.204836 32013 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:13.204993 32013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:13.205384 32013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:13.208693 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "1eee9641621049719a4c0d178f9299b0"
format_stamp: "Formatted at 2026-08-07 22:55:13 on dist-test-slave-5k54"
server_key: "9d82ca7985754aa66ec8c09e24ddc89e"
server_key_iv: "281153b482ee5cc01e74580df495b4c2"
server_key_version: "encryptionkey@0"
I20260807 22:55:13.209506 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "1eee9641621049719a4c0d178f9299b0"
format_stamp: "Formatted at 2026-08-07 22:55:13 on dist-test-slave-5k54"
server_key: "9d82ca7985754aa66ec8c09e24ddc89e"
server_key_iv: "281153b482ee5cc01e74580df495b4c2"
server_key_version: "encryptionkey@0"
I20260807 22:55:13.214819 32013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 22:55:13.218494 32029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:13.220093 32013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:13.220280 32013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "1eee9641621049719a4c0d178f9299b0"
format_stamp: "Formatted at 2026-08-07 22:55:13 on dist-test-slave-5k54"
server_key: "9d82ca7985754aa66ec8c09e24ddc89e"
server_key_iv: "281153b482ee5cc01e74580df495b4c2"
server_key_version: "encryptionkey@0"
I20260807 22:55:13.220494 32013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:13.249626 32013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:13.253813 32013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:13.254137 32013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:13.273427 32013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45981
I20260807 22:55:13.273470 32084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45981 every 8 connection(s)
I20260807 22:55:13.274866 32013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:13.278795 32085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:13.281193 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32013
I20260807 22:55:13.281394 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:13.281862 9995 external_mini_cluster.cc:1428] Setting key b7a8e053af5f608c44e2eab40ef7e2b4
I20260807 22:55:13.292872 32085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0: Bootstrap starting.
I20260807 22:55:13.297096 32085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:13.297551 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:13.283911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54947 (local address 127.0.0.1:45981)
0807 22:55:13.284813 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:13.284830 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:13.284885 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:55:13.287268 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:13.287296 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:13.287823 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:13.288263 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:13.288798 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:13.290859 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:13.292251 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:13.293274 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:13.293481 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:13.294687 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:13.294703 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:13.294723 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:13.295515 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:13.295613 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:13.295643 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:13.295803 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:13.296031 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 22:55:13.296214 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":347,"threads_started":1}
I20260807 22:55:13.298395 32085 log.cc:824] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:13.301483 32085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0: No bootstrap required, opened a new log
I20260807 22:55:13.301687 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:13.309664 32085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } }
I20260807 22:55:13.310083 32085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:13.310184 32085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eee9641621049719a4c0d178f9299b0, State: Initialized, Role: FOLLOWER
I20260807 22:55:13.310883 32085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } }
I20260807 22:55:13.311182 32085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:13.311350 32085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:13.311580 32085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:13.312212 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:13.306042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36261 (local address 127.0.0.1:45981)
0807 22:55:13.306238 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:13.306246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:13.306608 (+ 362us) server_negotiation.cc:407] Connection header received
0807 22:55:13.306835 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:13.306839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:13.306898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:13.307010 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:13.307526 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:13.308308 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:13.309614 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:13.310106 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:13.310256 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:13.311384 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:13.311389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:13.311392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:13.311408 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:13.311515 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:13.311541 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:13.311674 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:13.311871 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 22:55:13.311926 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:55:13.314348 32085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } }
I20260807 22:55:13.314886 32085 leader_election.cc:302] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eee9641621049719a4c0d178f9299b0; no voters:
I20260807 22:55:13.315514 32085 leader_election.cc:288] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:13.315644 32098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:13.317245 32098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [term 1 LEADER]: Becoming Leader. State: Replica: 1eee9641621049719a4c0d178f9299b0, State: Running, Role: LEADER
I20260807 22:55:13.317781 32085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:13.317737 32098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } }
I20260807 22:55:13.320401 32106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eee9641621049719a4c0d178f9299b0. Latest consensus state: current_term: 1 leader_uuid: "1eee9641621049719a4c0d178f9299b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } } }
I20260807 22:55:13.320763 32106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:13.320530 32101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eee9641621049719a4c0d178f9299b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eee9641621049719a4c0d178f9299b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45981 } } }
I20260807 22:55:13.320968 32101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:13.321707 32113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:13.334580 32113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:13.343887 32113 catalog_manager.cc:1347] Generated new cluster ID: d8a5f1b379d94e34b077411a5f950af2
I20260807 22:55:13.344008 32113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:13.388276 32113 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:13.390254 32113 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:13.403002 32113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eee9641621049719a4c0d178f9299b0: Generated new TSK 0
I20260807 22:55:13.404062 32113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:13.414148 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32013
W20260807 22:55:13.415303 32080 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54947
W20260807 22:55:13.416303 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32013: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:13.416502 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32013
W20260807 22:55:13.437288 31998 connection.cc:535] client connection to 127.0.0.1:45981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45981): Connection reset by peer (error 104)
2026-08-07T22:55:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2091 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:13 dist-test-slave-5k54 krb5kdc[32127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:13 dist-test-slave-5k54 krb5kdc[32127](info): set up 2 sockets
Aug 07 22:55:13 dist-test-slave-5k54 krb5kdc[32127](info): commencing operation
krb5kdc: starting...
W20260807 22:55:14.577466 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:14 dist-test-slave-5k54 krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:14.898499 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:15.240051 32142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:15.240335 32142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:15.240430 32142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:15.248495 32142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:15.248687 32142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:15.248790 32142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:15.248908 32142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:15.248976 32142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:15.259220 32142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:15.260540 32142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:15.261831 32142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:15.273413 32148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:15.273389 32151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:15.273478 32149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:15.274260 32142 server_base.cc:1034] running on GCE node
I20260807 22:55:15.275002 32142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:15.276391 32142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:15.277640 32142 hybrid_clock.cc:648] HybridClock initialized: now 1786143315277597 us; error 50 us; skew 500 ppm
I20260807 22:55:15.278069 32142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:15 dist-test-slave-5k54 krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143315, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:15.281314 32142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:15.282737 32142 webserver.cc:460] Webserver started at http://127.0.0.1:35839/ using document root <none> and password file <none>
I20260807 22:55:15.283380 32142 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:15.283588 32142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:15.283900 32142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:15.286900 32142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "454d8703e37f4730864ff0de7d19077c"
format_stamp: "Formatted at 2026-08-07 22:55:15 on dist-test-slave-5k54"
server_key: "28f8094b57d773d04d16128eee8cc7c3"
server_key_iv: "1769272a14771e673e42995bbb02be1d"
server_key_version: "encryptionkey@0"
I20260807 22:55:15.287720 32142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "454d8703e37f4730864ff0de7d19077c"
format_stamp: "Formatted at 2026-08-07 22:55:15 on dist-test-slave-5k54"
server_key: "28f8094b57d773d04d16128eee8cc7c3"
server_key_iv: "1769272a14771e673e42995bbb02be1d"
server_key_version: "encryptionkey@0"
I20260807 22:55:15.293052 32142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:55:15.296587 32159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:15.298101 32142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:55:15.298321 32142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "454d8703e37f4730864ff0de7d19077c"
format_stamp: "Formatted at 2026-08-07 22:55:15 on dist-test-slave-5k54"
server_key: "28f8094b57d773d04d16128eee8cc7c3"
server_key_iv: "1769272a14771e673e42995bbb02be1d"
server_key_version: "encryptionkey@0"
I20260807 22:55:15.298537 32142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:15.304728 32142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:15.308991 32142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:15.309309 32142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:15.329105 32142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43195
I20260807 22:55:15.329195 32218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43195 every 8 connection(s)
I20260807 22:55:15.330653 32142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:15.334492 32219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:15.339268 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32142
I20260807 22:55:15.339511 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:15.340003 9995 external_mini_cluster.cc:1428] Setting key 02d223617dfd59fa673c38a4c4a6ede9
I20260807 22:55:15.346616 32219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c: Bootstrap starting.
I20260807 22:55:15.349303 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:15.342098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49461 (local address 127.0.0.1:43195)
0807 22:55:15.342964 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:15.342979 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:15.343038 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:55:15.345502 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:15.345528 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:15.345954 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:15.346377 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:15.347366 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:15.347374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:15.347390 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:15.347406 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:15.347492 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:15.347527 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:15.347725 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:15.347850 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 22:55:15.348047 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":374,"threads_started":1}
I20260807 22:55:15.350715 32219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:15.352016 32219 log.cc:824] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:15.353295 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:15.355365 32219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c: No bootstrap required, opened a new log
I20260807 22:55:15.361485 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:15.358688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46299 (local address 127.0.0.1:43195)
0807 22:55:15.358947 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:15.358957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:15.359318 (+ 361us) server_negotiation.cc:407] Connection header received
0807 22:55:15.359568 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:15.359572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:15.359624 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:15.359714 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:15.360715 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:15.360719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:15.360723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:15.360739 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:15.360783 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:15.360803 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:15.360916 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:15.361058 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:55:15.361107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:55:15.364959 32219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260807 22:55:15.365356 32219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:15.365478 32219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454d8703e37f4730864ff0de7d19077c, State: Initialized, Role: FOLLOWER
I20260807 22:55:15.366055 32219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260807 22:55:15.366309 32219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:15.366451 32219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:15.366634 32219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:15.368993 32219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260807 22:55:15.369488 32219 leader_election.cc:302] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 454d8703e37f4730864ff0de7d19077c; no voters:
I20260807 22:55:15.370072 32219 leader_election.cc:288] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:15.370219 32229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:15.370961 32229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [term 1 LEADER]: Becoming Leader. State: Replica: 454d8703e37f4730864ff0de7d19077c, State: Running, Role: LEADER
I20260807 22:55:15.371703 32229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260807 22:55:15.372371 32219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:15.375063 32231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "454d8703e37f4730864ff0de7d19077c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260807 22:55:15.375511 32231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:15.375787 32232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 454d8703e37f4730864ff0de7d19077c. Latest consensus state: current_term: 1 leader_uuid: "454d8703e37f4730864ff0de7d19077c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454d8703e37f4730864ff0de7d19077c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260807 22:55:15.376163 32232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:15.377812 32240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:15.387059 32240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:15.396344 32240 catalog_manager.cc:1347] Generated new cluster ID: 7902b1800e064d5badd4d1522b9c0d7c
I20260807 22:55:15.396462 32240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:15.407963 32240 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:15.408507 32240 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:15.422367 32240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 454d8703e37f4730864ff0de7d19077c: Generated new TSK 0
I20260807 22:55:15.423416 32240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:15.434620 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32142
W20260807 22:55:15.435968 32213 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49461
W20260807 22:55:15.436791 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32142: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:15.436961 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32142
2026-08-07T22:55:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:15 dist-test-slave-5k54 krb5kdc[32255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:15 dist-test-slave-5k54 krb5kdc[32255](info): set up 2 sockets
Aug 07 22:55:15 dist-test-slave-5k54 krb5kdc[32255](info): commencing operation
krb5kdc: starting...
W20260807 22:55:16.593421 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:16 dist-test-slave-5k54 krb5kdc[32255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:16.916740 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:17.266458 32268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:17.266808 32268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:17.266981 32268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:17.274322 32268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:17.274487 32268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:17.274613 32268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:17.274716 32268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:17.274811 32268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:17.284934 32268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:17.286280 32268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:17.287710 32268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:17.299610 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:17.299551 32273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:17.300248 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:17.300704 32268 server_base.cc:1034] running on GCE node
I20260807 22:55:17.301487 32268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:17.302946 32268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:17.304215 32268 hybrid_clock.cc:648] HybridClock initialized: now 1786143317304173 us; error 57 us; skew 500 ppm
I20260807 22:55:17.304656 32268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:17 dist-test-slave-5k54 krb5kdc[32255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143317, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:17.308147 32268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:17.309739 32268 webserver.cc:460] Webserver started at http://127.0.0.1:38837/ using document root <none> and password file <none>
I20260807 22:55:17.310503 32268 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:17.310664 32268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:17.311008 32268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:17.314307 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "36d6355301354ff19a7c472bd9ece070"
format_stamp: "Formatted at 2026-08-07 22:55:17 on dist-test-slave-5k54"
server_key: "de8b85338d5259dd643d3ad120517fb2"
server_key_iv: "408b0fcd956ad213322b1679c17db3f3"
server_key_version: "encryptionkey@0"
I20260807 22:55:17.315402 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "36d6355301354ff19a7c472bd9ece070"
format_stamp: "Formatted at 2026-08-07 22:55:17 on dist-test-slave-5k54"
server_key: "de8b85338d5259dd643d3ad120517fb2"
server_key_iv: "408b0fcd956ad213322b1679c17db3f3"
server_key_version: "encryptionkey@0"
I20260807 22:55:17.320647 32268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:55:17.324101 32283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:17.325441 32268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:17.325635 32268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "36d6355301354ff19a7c472bd9ece070"
format_stamp: "Formatted at 2026-08-07 22:55:17 on dist-test-slave-5k54"
server_key: "de8b85338d5259dd643d3ad120517fb2"
server_key_iv: "408b0fcd956ad213322b1679c17db3f3"
server_key_version: "encryptionkey@0"
I20260807 22:55:17.325845 32268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:17.340770 32268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:17.344671 32268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:17.344985 32268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:17.364107 32268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41511
I20260807 22:55:17.364143 32336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41511 every 8 connection(s)
I20260807 22:55:17.365541 32268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:17.368194 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32268
I20260807 22:55:17.368376 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:17.368836 9995 external_mini_cluster.cc:1428] Setting key f4a1af19a77873f74e1710fb0a7b5598
I20260807 22:55:17.369712 32337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:55:17 dist-test-slave-5k54 krb5kdc[32255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:55:17.384486 32337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070: Bootstrap starting.
I20260807 22:55:17.388983 32337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:17.390251 32337 log.cc:824] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:17.393355 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:17.370792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46889 (local address 127.0.0.1:41511)
0807 22:55:17.371644 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:17.371662 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:17.371714 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:55:17.373546 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:17.373575 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:17.374127 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:17.374605 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:17.376116 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:17.378234 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:17.379521 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:17.380568 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:17.380757 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:17.384943 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:17.384954 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:17.384973 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:17.385029 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:17.389093 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:17.389819 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:17.389827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:17.389835 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:17.390072 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:17.390437 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:17.390443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:17.390448 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:17.391215 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:17.391582 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:17.391981 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:55:17.392354 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":340,"threads_started":1}
I20260807 22:55:17.393543 32337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070: No bootstrap required, opened a new log
I20260807 22:55:17.397082 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:17.403908 32337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } }
I20260807 22:55:17.404400 32337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:17.404556 32337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d6355301354ff19a7c472bd9ece070, State: Initialized, Role: FOLLOWER
I20260807 22:55:17.405294 32337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } }
I20260807 22:55:17.405596 32337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:17.405766 32337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:17.405959 32337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:17.409098 32337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } }
I20260807 22:55:17.409719 32337 leader_election.cc:302] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36d6355301354ff19a7c472bd9ece070; no voters:
I20260807 22:55:17.410409 32337 leader_election.cc:288] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:17.410516 32348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:17.412736 32337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:17.412079 32348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [term 1 LEADER]: Becoming Leader. State: Replica: 36d6355301354ff19a7c472bd9ece070, State: Running, Role: LEADER
I20260807 22:55:17.414031 32348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } }
I20260807 22:55:17.417266 32350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d6355301354ff19a7c472bd9ece070. Latest consensus state: current_term: 1 leader_uuid: "36d6355301354ff19a7c472bd9ece070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } } }
I20260807 22:55:17.417709 32350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:17.417642 32349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36d6355301354ff19a7c472bd9ece070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d6355301354ff19a7c472bd9ece070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41511 } } }
I20260807 22:55:17.417955 32349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:17.421360 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:17.402474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50321 (local address 127.0.0.1:41511)
0807 22:55:17.402710 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:17.402716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:17.403089 (+ 373us) server_negotiation.cc:407] Connection header received
0807 22:55:17.403254 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:17.403257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:17.403316 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:17.403396 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:17.405383 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:17.406342 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:17.408184 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:17.408630 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:17.408721 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:17.411926 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:17.411941 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:17.411946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:17.411986 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:17.416051 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:17.417111 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:17.417120 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:17.417124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:17.417342 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:17.417853 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:17.417866 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:17.417872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:17.418468 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:17.418825 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:17.420501 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0807 22:55:17.420804 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:55:17.424835 32358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:17.429951 32358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:17.439074 32358 catalog_manager.cc:1347] Generated new cluster ID: 85894a6651184af3b0f2540e43dfcab6
I20260807 22:55:17.439203 32358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:17.447716 32358 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:17.449230 32358 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:17.455189 32358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36d6355301354ff19a7c472bd9ece070: Generated new TSK 0
I20260807 22:55:17.456171 32358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:17.469502 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32268
I20260807 22:55:17.470698 32332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:17.562410 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32268
2026-08-07T22:55:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2125 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:17 dist-test-slave-5k54 krb5kdc[32371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:17 dist-test-slave-5k54 krb5kdc[32371](info): set up 2 sockets
Aug 07 22:55:17 dist-test-slave-5k54 krb5kdc[32371](info): commencing operation
krb5kdc: starting...
W20260807 22:55:18.736238 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:18 dist-test-slave-5k54 krb5kdc[32371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:19.067273 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:19.412309 32384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:19.412590 32384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:19.412758 32384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:19.419953 32384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:19.420056 32384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:19.420176 32384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:19.420287 32384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:19.420372 32384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:19.430143 32384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:19.431481 32384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:19.432837 32384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:19.444630 32389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:19.444720 32391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:19.444680 32393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:19.445830 32384 server_base.cc:1034] running on GCE node
I20260807 22:55:19.446615 32384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:19.448081 32384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:19.449314 32384 hybrid_clock.cc:648] HybridClock initialized: now 1786143319449274 us; error 39 us; skew 500 ppm
I20260807 22:55:19.449662 32384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:19 dist-test-slave-5k54 krb5kdc[32371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:19.457382 32384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:19.458925 32384 webserver.cc:460] Webserver started at http://127.0.0.1:38965/ using document root <none> and password file <none>
I20260807 22:55:19.459709 32384 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:19.459856 32384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:19.460186 32384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:19.463166 32384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "0bcf3a7eb80b486aad3463c316814b15"
format_stamp: "Formatted at 2026-08-07 22:55:19 on dist-test-slave-5k54"
server_key: "b305cb9ddd0e1ee5125a47d32d69950e"
server_key_iv: "08f81ca8ece330dfd3c1f00b4ec00bfc"
server_key_version: "encryptionkey@0"
I20260807 22:55:19.463992 32384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "0bcf3a7eb80b486aad3463c316814b15"
format_stamp: "Formatted at 2026-08-07 22:55:19 on dist-test-slave-5k54"
server_key: "b305cb9ddd0e1ee5125a47d32d69950e"
server_key_iv: "08f81ca8ece330dfd3c1f00b4ec00bfc"
server_key_version: "encryptionkey@0"
I20260807 22:55:19.469648 32384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 22:55:19.473208 32400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:19.474587 32384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:55:19.474773 32384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "0bcf3a7eb80b486aad3463c316814b15"
format_stamp: "Formatted at 2026-08-07 22:55:19 on dist-test-slave-5k54"
server_key: "b305cb9ddd0e1ee5125a47d32d69950e"
server_key_iv: "08f81ca8ece330dfd3c1f00b4ec00bfc"
server_key_version: "encryptionkey@0"
I20260807 22:55:19.474975 32384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:19.494860 32384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:19.498792 32384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:19.499089 32384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:19.518790 32384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44153
I20260807 22:55:19.518843 32462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44153 every 8 connection(s)
I20260807 22:55:19.520243 32384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:19.524092 32463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:19.529675 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32384
I20260807 22:55:19.529862 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:19.530426 9995 external_mini_cluster.cc:1428] Setting key 992fe1b7f72434cf38706df90743bf24
I20260807 22:55:19.535182 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15: Bootstrap starting.
I20260807 22:55:19.538892 32463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:19.540061 32463 log.cc:824] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:19.542809 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15: No bootstrap required, opened a new log
I20260807 22:55:19.543910 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:19.532237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39713 (local address 127.0.0.1:44153)
0807 22:55:19.533079 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:19.533090 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:19.533130 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:55:19.534757 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:19.534779 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:19.535173 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:19.535605 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:19.536162 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:19.537455 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:19.539415 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:19.540163 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:19.540373 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:19.541561 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:19.541569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:19.541582 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:19.542103 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:19.542176 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:19.542199 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:19.542327 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:19.542575 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 22:55:19.542850 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":402,"threads_started":1}
I20260807 22:55:19.547467 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:19.551034 32463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } }
I20260807 22:55:19.551375 32463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:19.551539 32463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bcf3a7eb80b486aad3463c316814b15, State: Initialized, Role: FOLLOWER
I20260807 22:55:19.552544 32463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } }
I20260807 22:55:19.552842 32463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:19.553009 32463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:19.553217 32463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:19.556324 32463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } }
I20260807 22:55:19.556823 32463 leader_election.cc:302] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bcf3a7eb80b486aad3463c316814b15; no voters:
I20260807 22:55:19.557585 32463 leader_election.cc:288] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:19.557654 32474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:19.559013 32474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [term 1 LEADER]: Becoming Leader. State: Replica: 0bcf3a7eb80b486aad3463c316814b15, State: Running, Role: LEADER
I20260807 22:55:19.559099 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:19.551714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56067 (local address 127.0.0.1:44153)
0807 22:55:19.551933 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:19.551941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:19.552350 (+ 409us) server_negotiation.cc:407] Connection header received
0807 22:55:19.552559 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:19.552562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:19.552617 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:19.552691 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:19.553318 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:19.554051 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:19.555984 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:19.556422 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:19.556564 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:19.557793 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:19.557800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:19.557805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:19.557826 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:19.557880 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:19.557908 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:19.558029 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:19.558543 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 22:55:19.558634 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:55:19.559769 32474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } }
I20260807 22:55:19.560002 32463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:19.562579 32476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bcf3a7eb80b486aad3463c316814b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } } }
I20260807 22:55:19.562675 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bcf3a7eb80b486aad3463c316814b15. Latest consensus state: current_term: 1 leader_uuid: "0bcf3a7eb80b486aad3463c316814b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bcf3a7eb80b486aad3463c316814b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44153 } } }
I20260807 22:55:19.562934 32476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:19.563025 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:19.565058 32481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:19.569753 32481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:19.580850 32481 catalog_manager.cc:1347] Generated new cluster ID: baca4b59ec044cf7914c40c218382457
I20260807 22:55:19.580974 32481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:19.600065 32481 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:19.602007 32481 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:19.613721 32481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bcf3a7eb80b486aad3463c316814b15: Generated new TSK 0
I20260807 22:55:19.614777 32481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:19.627928 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32384
W20260807 22:55:19.629267 32457 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39713
W20260807 22:55:19.630079 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32384: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:19.630244 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32384
2026-08-07T22:55:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2069 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:19 dist-test-slave-5k54 krb5kdc[32499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:19 dist-test-slave-5k54 krb5kdc[32499](info): set up 2 sockets
Aug 07 22:55:19 dist-test-slave-5k54 krb5kdc[32499](info): commencing operation
krb5kdc: starting...
W20260807 22:55:20.800896 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:21 dist-test-slave-5k54 krb5kdc[32499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:21.123561 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:21.463789 32514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:21.464150 32514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:21.464329 32514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:21.471642 32514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:21.471801 32514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:21.471925 32514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:21.472045 32514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:21.472136 32514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:21.481997 32514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:21.483376 32514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:21.484719 32514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:21.496816 32520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:21.496976 32521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:21.496994 32524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:21.497681 32514 server_base.cc:1034] running on GCE node
I20260807 22:55:21.498735 32514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:21.500406 32514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:21.501703 32514 hybrid_clock.cc:648] HybridClock initialized: now 1786143321501655 us; error 32 us; skew 500 ppm
I20260807 22:55:21.502138 32514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:21 dist-test-slave-5k54 krb5kdc[32499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:21.505506 32514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:21.507040 32514 webserver.cc:460] Webserver started at http://127.0.0.1:42957/ using document root <none> and password file <none>
I20260807 22:55:21.507824 32514 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:21.507982 32514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:21.508293 32514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:21.511327 32514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "0e742f40cb0c4d0fbf92e2437600897b"
format_stamp: "Formatted at 2026-08-07 22:55:21 on dist-test-slave-5k54"
server_key: "a8ca06e8c77234f8f53e65d19b0ae36d"
server_key_iv: "cc63727ab62efe1617f2082761bf30e6"
server_key_version: "encryptionkey@0"
I20260807 22:55:21.512213 32514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "0e742f40cb0c4d0fbf92e2437600897b"
format_stamp: "Formatted at 2026-08-07 22:55:21 on dist-test-slave-5k54"
server_key: "a8ca06e8c77234f8f53e65d19b0ae36d"
server_key_iv: "cc63727ab62efe1617f2082761bf30e6"
server_key_version: "encryptionkey@0"
I20260807 22:55:21.517457 32514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:55:21.521045 32533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:21.522436 32514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:55:21.522620 32514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "0e742f40cb0c4d0fbf92e2437600897b"
format_stamp: "Formatted at 2026-08-07 22:55:21 on dist-test-slave-5k54"
server_key: "a8ca06e8c77234f8f53e65d19b0ae36d"
server_key_iv: "cc63727ab62efe1617f2082761bf30e6"
server_key_version: "encryptionkey@0"
I20260807 22:55:21.522825 32514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:21.528687 32514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:21.532693 32514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:21.532998 32514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:21.551668 32514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33235
I20260807 22:55:21.551721 32591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33235 every 8 connection(s)
I20260807 22:55:21.553097 32514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:21.553591 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32514
I20260807 22:55:21.553784 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:21.554440 9995 external_mini_cluster.cc:1428] Setting key 82e02cc2ed581ed2df144ffbb120c947
I20260807 22:55:21.557914 32592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:21.563383 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:21.556339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43525 (local address 127.0.0.1:33235)
0807 22:55:21.557179 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:21.557194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:21.557243 (+ 49us) server_negotiation.cc:407] Connection header received
0807 22:55:21.559584 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:21.559611 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:21.560028 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:21.560430 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:21.561457 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:21.561464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:21.561479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:21.561497 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:21.561591 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:21.561615 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:21.561828 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:21.561966 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:55:21.562166 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":355,"threads_started":1}
I20260807 22:55:21.567510 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:21.572316 32592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b: Bootstrap starting.
I20260807 22:55:21.575481 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:21.572946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54227 (local address 127.0.0.1:33235)
0807 22:55:21.573124 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:21.573130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:21.573517 (+ 387us) server_negotiation.cc:407] Connection header received
0807 22:55:21.573706 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:21.573711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:21.573753 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:21.573856 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:21.574785 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:21.574792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:21.574800 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:21.574814 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:21.574850 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:21.574870 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:21.574986 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:21.575111 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 22:55:21.575169 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:55:21.576077 32592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:21.577343 32592 log.cc:824] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:21.579959 32592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b: No bootstrap required, opened a new log
I20260807 22:55:21.587437 32592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260807 22:55:21.587908 32592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:21.588030 32592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e742f40cb0c4d0fbf92e2437600897b, State: Initialized, Role: FOLLOWER
I20260807 22:55:21.588689 32592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260807 22:55:21.588892 32592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:21.589053 32592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:21.589234 32592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:21.591411 32592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260807 22:55:21.591992 32592 leader_election.cc:302] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e742f40cb0c4d0fbf92e2437600897b; no voters:
I20260807 22:55:21.592626 32592 leader_election.cc:288] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:21.592801 32604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:21.593494 32604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [term 1 LEADER]: Becoming Leader. State: Replica: 0e742f40cb0c4d0fbf92e2437600897b, State: Running, Role: LEADER
I20260807 22:55:21.594220 32604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260807 22:55:21.594985 32592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:21.597611 32606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e742f40cb0c4d0fbf92e2437600897b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } } }
I20260807 22:55:21.597640 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e742f40cb0c4d0fbf92e2437600897b. Latest consensus state: current_term: 1 leader_uuid: "0e742f40cb0c4d0fbf92e2437600897b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e742f40cb0c4d0fbf92e2437600897b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } } }
I20260807 22:55:21.598291 32606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:21.598291 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:21.604110 32614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:21.610101 32614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:21.619006 32614 catalog_manager.cc:1347] Generated new cluster ID: b02e89b7a472409cb1da086d418ebc8f
I20260807 22:55:21.619168 32614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:21.648777 32614 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:21.649322 32614 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:21.667044 32614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e742f40cb0c4d0fbf92e2437600897b: Generated new TSK 0
I20260807 22:55:21.668190 32614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:21.681083 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32514
W20260807 22:55:21.682161 32587 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43525
W20260807 22:55:21.682902 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32514: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:21.683056 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32514
2026-08-07T22:55:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2048 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:21 dist-test-slave-5k54 krb5kdc[32628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:21 dist-test-slave-5k54 krb5kdc[32628](info): set up 2 sockets
Aug 07 22:55:21 dist-test-slave-5k54 krb5kdc[32628](info): commencing operation
krb5kdc: starting...
W20260807 22:55:22.857364 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:23.179848 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:23.529772 32644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:23.530123 32644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:23.530295 32644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:23.537576 32644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:23.537736 32644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:23.537848 32644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:23.537958 32644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:23.538044 32644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:23.548036 32644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:23.549541 32644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:23.550910 32644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:23.562999 32649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:23.562923 32652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:23.562963 32650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:23.563954 32644 server_base.cc:1034] running on GCE node
I20260807 22:55:23.564720 32644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:23.566111 32644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:23.567344 32644 hybrid_clock.cc:648] HybridClock initialized: now 1786143323567304 us; error 50 us; skew 500 ppm
I20260807 22:55:23.567713 32644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:23.570809 32644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:23.572355 32644 webserver.cc:460] Webserver started at http://127.0.0.1:38725/ using document root <none> and password file <none>
I20260807 22:55:23.573045 32644 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:23.573199 32644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:23.573510 32644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:23.576525 32644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "85fc67173aa94ce388d20aa14db67213"
format_stamp: "Formatted at 2026-08-07 22:55:23 on dist-test-slave-5k54"
server_key: "9417aba7aaa208a863e6c8296e6e69cf"
server_key_iv: "ef67b0cf943ae260f9d374d4a44e8795"
server_key_version: "encryptionkey@0"
I20260807 22:55:23.577296 32644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "85fc67173aa94ce388d20aa14db67213"
format_stamp: "Formatted at 2026-08-07 22:55:23 on dist-test-slave-5k54"
server_key: "9417aba7aaa208a863e6c8296e6e69cf"
server_key_iv: "ef67b0cf943ae260f9d374d4a44e8795"
server_key_version: "encryptionkey@0"
I20260807 22:55:23.582377 32644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:23.585906 32660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:23.587168 32644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 22:55:23.587348 32644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "85fc67173aa94ce388d20aa14db67213"
format_stamp: "Formatted at 2026-08-07 22:55:23 on dist-test-slave-5k54"
server_key: "9417aba7aaa208a863e6c8296e6e69cf"
server_key_iv: "ef67b0cf943ae260f9d374d4a44e8795"
server_key_version: "encryptionkey@0"
I20260807 22:55:23.587603 32644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:23.611373 32644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:23.615621 32644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:23.615929 32644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:23.634763 32644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46631
I20260807 22:55:23.634799 32724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46631 every 8 connection(s)
I20260807 22:55:23.636276 32644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:23.640067 32726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:23.640920 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 32644
I20260807 22:55:23.641112 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:23.641566 9995 external_mini_cluster.cc:1428] Setting key be3d818d8088228249cce203444443e5
I20260807 22:55:23.654045 32726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213: Bootstrap starting.
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:55:23.657361 32726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:23.658747 32726 log.cc:824] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:23.662264 32726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213: No bootstrap required, opened a new log
I20260807 22:55:23.666953 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:23.643399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:58541 (local address 127.0.0.1:46631)
0807 22:55:23.644233 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:23.644245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:23.644285 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:55:23.646109 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:23.646131 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:23.646548 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:23.646936 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:23.648730 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:23.650739 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:23.652086 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:23.653213 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:23.653398 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:23.657924 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:23.657935 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:23.657954 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:23.658001 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:23.662034 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:23.663164 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:23.663172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:23.663179 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:23.663405 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:23.663860 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:23.663865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:23.663868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:23.664569 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:23.664907 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:23.665250 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:55:23.665582 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":364,"threads_started":1}
I20260807 22:55:23.670869 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:23.672048 32726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260807 22:55:23.672483 32726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:23.672583 32726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fc67173aa94ce388d20aa14db67213, State: Initialized, Role: FOLLOWER
I20260807 22:55:23.673182 32726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260807 22:55:23.673478 32726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:23.673623 32726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:23.673796 32726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:23.677145 32726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260807 22:55:23.677744 32726 leader_election.cc:302] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85fc67173aa94ce388d20aa14db67213; no voters:
I20260807 22:55:23.678464 32726 leader_election.cc:288] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:23.678560 32737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:23.681064 32726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:23.685581 32737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [term 1 LEADER]: Becoming Leader. State: Replica: 85fc67173aa94ce388d20aa14db67213, State: Running, Role: LEADER
I20260807 22:55:23.686354 32737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260807 22:55:23.690121 32739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85fc67173aa94ce388d20aa14db67213. Latest consensus state: current_term: 1 leader_uuid: "85fc67173aa94ce388d20aa14db67213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } } }
I20260807 22:55:23.690101 32738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85fc67173aa94ce388d20aa14db67213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc67173aa94ce388d20aa14db67213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } } }
I20260807 22:55:23.690825 32738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:23.690825 32739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:23.693725 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:23.676468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:35991 (local address 127.0.0.1:46631)
0807 22:55:23.676649 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:23.676655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:23.677208 (+ 553us) server_negotiation.cc:407] Connection header received
0807 22:55:23.677318 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:23.677324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:23.677398 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:23.677531 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:23.679269 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:23.680219 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:23.681624 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:23.682105 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:23.682239 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:23.684973 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:23.684978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:23.684982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:23.685014 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:23.688481 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:23.689596 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:23.689602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:23.689608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:23.689748 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:23.690369 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:23.690376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:23.690381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:23.691201 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:23.691555 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:23.692917 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0807 22:55:23.693154 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:55:23.703728 32747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:23.708765 32747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:23.717975 32747 catalog_manager.cc:1347] Generated new cluster ID: 0e41c49019f5458785d8576a964eae11
I20260807 22:55:23.718094 32747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:23.751065 32747 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:23.752508 32747 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:23.759135 32747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85fc67173aa94ce388d20aa14db67213: Generated new TSK 0
I20260807 22:55:23.760164 32747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:23.777961 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 32644
I20260807 22:55:23.779703 32720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:23.873799 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 32644
W20260807 22:55:23.897727 32626 connection.cc:535] client connection to 127.0.0.1:46631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46631): Connection reset by peer (error 104)
2026-08-07T22:55:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32762](info): set up 2 sockets
Aug 07 22:55:23 dist-test-slave-5k54 krb5kdc[32762](info): commencing operation
krb5kdc: starting...
W20260807 22:55:25.069096 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:25 dist-test-slave-5k54 krb5kdc[32762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:25.402796 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:25.764014 307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:25.764354 307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:25.764526 307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:25.771885 307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:25.772037 307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:25.772140 307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:25.772246 307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:25.772322 307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:25.782279 307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:25.783766 307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:25.785194 307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:25.798134 313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:25.798147 316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:25.798061 314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:25.798924 307 server_base.cc:1034] running on GCE node
I20260807 22:55:25.799888 307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:25.801422 307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:25.802680 307 hybrid_clock.cc:648] HybridClock initialized: now 1786143325802641 us; error 34 us; skew 500 ppm
I20260807 22:55:25.803100 307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:25 dist-test-slave-5k54 krb5kdc[32762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:25.812441 307 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:25.814360 307 webserver.cc:460] Webserver started at http://127.0.0.1:38281/ using document root <none> and password file <none>
I20260807 22:55:25.815161 307 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:25.815320 307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:25.815681 307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:25.818725 307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "ab7dff670853470fb75cfc53b9032930"
format_stamp: "Formatted at 2026-08-07 22:55:25 on dist-test-slave-5k54"
server_key: "2771f407fe786f527ed2d5feb7e68061"
server_key_iv: "a1363486c3871cb65cea2125fbb11600"
server_key_version: "encryptionkey@0"
I20260807 22:55:25.819572 307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "ab7dff670853470fb75cfc53b9032930"
format_stamp: "Formatted at 2026-08-07 22:55:25 on dist-test-slave-5k54"
server_key: "2771f407fe786f527ed2d5feb7e68061"
server_key_iv: "a1363486c3871cb65cea2125fbb11600"
server_key_version: "encryptionkey@0"
I20260807 22:55:25.825102 307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 22:55:25.828753 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:25.830159 307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:25.830343 307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "ab7dff670853470fb75cfc53b9032930"
format_stamp: "Formatted at 2026-08-07 22:55:25 on dist-test-slave-5k54"
server_key: "2771f407fe786f527ed2d5feb7e68061"
server_key_iv: "a1363486c3871cb65cea2125fbb11600"
server_key_version: "encryptionkey@0"
I20260807 22:55:25.830521 307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:25.863988 307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:25.868360 307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:25.868664 307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:25.888103 307 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45519
I20260807 22:55:25.888177 381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45519 every 8 connection(s)
I20260807 22:55:25.889514 307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:25.893304 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:25.896050 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 307
I20260807 22:55:25.896247 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:25.896833 9995 external_mini_cluster.cc:1428] Setting key 0d5bde2dd452457854f8ffd49dccaa4b
I20260807 22:55:25.905599 382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930: Bootstrap starting.
I20260807 22:55:25.909240 382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:25.910431 382 log.cc:824] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:25.910614 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:25.898669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:49183 (local address 127.0.0.1:45519)
0807 22:55:25.899553 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:25.899564 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:25.899605 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:55:25.901275 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:25.901297 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:25.901681 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:25.902021 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:25.902649 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:25.904100 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:25.905997 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:25.906692 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:25.906883 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:25.908267 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:25.908275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:25.908288 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:25.908810 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:25.908901 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:25.908925 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:25.909046 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:25.909398 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 22:55:25.909571 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":399,"threads_started":1}
I20260807 22:55:25.913544 382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930: No bootstrap required, opened a new log
I20260807 22:55:25.914686 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:25.924154 382 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } }
I20260807 22:55:25.924559 382 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:25.924724 382 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7dff670853470fb75cfc53b9032930, State: Initialized, Role: FOLLOWER
I20260807 22:55:25.925364 382 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } }
I20260807 22:55:25.925633 382 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:25.925798 382 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:25.925990 382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:25.927579 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:25.920228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:59263 (local address 127.0.0.1:45519)
0807 22:55:25.920433 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:25.920440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:25.920945 (+ 505us) server_negotiation.cc:407] Connection header received
0807 22:55:25.921112 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:25.921118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:25.921201 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:25.921307 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:25.921932 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:25.923148 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:25.924666 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:25.925340 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:25.925487 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:25.926735 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:25.926741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:25.926748 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:25.926767 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:25.926807 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:25.926827 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:25.926972 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:25.927167 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 22:55:25.927230 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:55:25.929009 382 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } }
I20260807 22:55:25.929569 382 leader_election.cc:302] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab7dff670853470fb75cfc53b9032930; no voters:
I20260807 22:55:25.930137 382 leader_election.cc:288] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:25.930291 392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:25.931097 392 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [term 1 LEADER]: Becoming Leader. State: Replica: ab7dff670853470fb75cfc53b9032930, State: Running, Role: LEADER
I20260807 22:55:25.931703 392 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } }
I20260807 22:55:25.932284 382 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:25.935520 394 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab7dff670853470fb75cfc53b9032930. Latest consensus state: current_term: 1 leader_uuid: "ab7dff670853470fb75cfc53b9032930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } } }
I20260807 22:55:25.935468 393 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab7dff670853470fb75cfc53b9032930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7dff670853470fb75cfc53b9032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45519 } } }
I20260807 22:55:25.935993 394 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:25.936005 393 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:25.938809 399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:25.944427 399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:25.954434 399 catalog_manager.cc:1347] Generated new cluster ID: 99ebda52cdb242e19577def4f766d966
I20260807 22:55:25.954566 399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:25.999415 399 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:26.000771 399 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:26.007099 399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab7dff670853470fb75cfc53b9032930: Generated new TSK 0
I20260807 22:55:26.008033 399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:26.029382 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 307
W20260807 22:55:26.030593 376 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:49183
W20260807 22:55:26.031450 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 307: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:26.031613 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 307
W20260807 22:55:26.051748 32760 connection.cc:535] client connection to 127.0.0.1:45519 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45519): Connection reset by peer (error 104)
2026-08-07T22:55:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:26 dist-test-slave-5k54 krb5kdc[413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:26 dist-test-slave-5k54 krb5kdc[413](info): set up 2 sockets
Aug 07 22:55:26 dist-test-slave-5k54 krb5kdc[413](info): commencing operation
krb5kdc: starting...
W20260807 22:55:27.205122 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:27 dist-test-slave-5k54 krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:27.529110 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:27.873255 426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:27.873543 426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:27.873731 426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:27.880983 426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:27.881088 426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:27.881213 426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:27.881318 426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:27.881389 426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:27.892079 426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:27.893385 426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:27.894688 426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:27.906893 435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:27.906978 438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:27.906945 434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:27.908048 426 server_base.cc:1034] running on GCE node
I20260807 22:55:27.908885 426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:27.910211 426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:27.911490 426 hybrid_clock.cc:648] HybridClock initialized: now 1786143327911408 us; error 52 us; skew 500 ppm
I20260807 22:55:27.911834 426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:27 dist-test-slave-5k54 krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:27.915030 426 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:27.916599 426 webserver.cc:460] Webserver started at http://127.0.0.1:40403/ using document root <none> and password file <none>
I20260807 22:55:27.917276 426 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:27.917428 426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:27.917737 426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:27.920768 426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "ef8303d477884d4383ab17639aff4e6e"
format_stamp: "Formatted at 2026-08-07 22:55:27 on dist-test-slave-5k54"
server_key: "785d6570e3ad01d1d021ed73c3697369"
server_key_iv: "49ab585a5a05159e89db68fe26408e86"
server_key_version: "encryptionkey@0"
I20260807 22:55:27.921533 426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "ef8303d477884d4383ab17639aff4e6e"
format_stamp: "Formatted at 2026-08-07 22:55:27 on dist-test-slave-5k54"
server_key: "785d6570e3ad01d1d021ed73c3697369"
server_key_iv: "49ab585a5a05159e89db68fe26408e86"
server_key_version: "encryptionkey@0"
I20260807 22:55:27.926549 426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 22:55:27.929884 446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:27.931138 426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:55:27.931319 426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "ef8303d477884d4383ab17639aff4e6e"
format_stamp: "Formatted at 2026-08-07 22:55:27 on dist-test-slave-5k54"
server_key: "785d6570e3ad01d1d021ed73c3697369"
server_key_iv: "49ab585a5a05159e89db68fe26408e86"
server_key_version: "encryptionkey@0"
I20260807 22:55:27.931593 426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:27.937489 426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:27.941556 426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:27.941865 426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:27.960908 426 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35149
I20260807 22:55:27.960947 499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35149 every 8 connection(s)
I20260807 22:55:27.962371 426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:27.966089 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:27.969256 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 426
I20260807 22:55:27.969455 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:27.969897 9995 external_mini_cluster.cc:1428] Setting key 52774f5ac9872bfbfa0bc759e9435943
I20260807 22:55:27.979334 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:27.971760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:46853 (local address 127.0.0.1:35149)
0807 22:55:27.972587 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:27.972602 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:27.972649 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:55:27.974999 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:27.975030 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:27.975485 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:27.975965 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:27.977389 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:27.977397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:27.977413 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:27.977440 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:27.977515 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:27.977546 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:27.977769 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:27.977907 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:55:27.978114 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":323,"threads_started":1}
I20260807 22:55:27.981040 500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e: Bootstrap starting.
I20260807 22:55:27.984079 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:27.985268 500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:27.986464 500 log.cc:824] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:27.989523 500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e: No bootstrap required, opened a new log
I20260807 22:55:27.996704 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:27.993990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:57719 (local address 127.0.0.1:35149)
0807 22:55:27.994145 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:27.994151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:27.994418 (+ 267us) server_negotiation.cc:407] Connection header received
0807 22:55:27.994689 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:27.994695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:27.994759 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:27.994928 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:27.995962 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:27.995968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:27.995972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:27.995991 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:27.996053 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:27.996076 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:27.996196 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:27.996291 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 22:55:27.996353 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:55:28.000236 500 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } }
I20260807 22:55:28.000629 500 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:28.000733 500 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8303d477884d4383ab17639aff4e6e, State: Initialized, Role: FOLLOWER
I20260807 22:55:28.001312 500 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } }
I20260807 22:55:28.001561 500 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:28.001719 500 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:28.001901 500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:28.005065 500 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } }
I20260807 22:55:28.005762 500 leader_election.cc:302] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8303d477884d4383ab17639aff4e6e; no voters:
I20260807 22:55:28.006381 500 leader_election.cc:288] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:28.006508 511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:28.007261 511 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [term 1 LEADER]: Becoming Leader. State: Replica: ef8303d477884d4383ab17639aff4e6e, State: Running, Role: LEADER
I20260807 22:55:28.007906 511 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } }
I20260807 22:55:28.008739 500 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:28.012782 513 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8303d477884d4383ab17639aff4e6e. Latest consensus state: current_term: 1 leader_uuid: "ef8303d477884d4383ab17639aff4e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } } }
I20260807 22:55:28.012830 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8303d477884d4383ab17639aff4e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8303d477884d4383ab17639aff4e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35149 } } }
I20260807 22:55:28.013321 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:28.013348 513 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:28.020520 516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:28.026230 516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:28.035861 516 catalog_manager.cc:1347] Generated new cluster ID: 32a2fcfbaf9c4a18afc52b3173829877
I20260807 22:55:28.036016 516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:28.062858 516 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:28.063349 516 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:28.071256 516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8303d477884d4383ab17639aff4e6e: Generated new TSK 0
I20260807 22:55:28.072069 516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:28.079067 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 426
W20260807 22:55:28.080534 495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:46853
W20260807 22:55:28.081223 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 426: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:28.081393 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 426
2026-08-07T22:55:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:28 dist-test-slave-5k54 krb5kdc[533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:28 dist-test-slave-5k54 krb5kdc[533](info): set up 2 sockets
Aug 07 22:55:28 dist-test-slave-5k54 krb5kdc[533](info): commencing operation
krb5kdc: starting...
W20260807 22:55:29.245358 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:29 dist-test-slave-5k54 krb5kdc[533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:29.567993 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:29.905440 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:29.905726 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:29.905915 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:29.913197 548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:29.913304 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:29.913414 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:29.913522 548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:29.913605 548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:29.923507 548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:29.924904 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:29.926245 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:29.938409 554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:29.938337 557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:29.938441 555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:29.939512 548 server_base.cc:1034] running on GCE node
I20260807 22:55:29.940335 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:29.941709 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:29.942992 548 hybrid_clock.cc:648] HybridClock initialized: now 1786143329942913 us; error 69 us; skew 500 ppm
I20260807 22:55:29.943347 548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:29 dist-test-slave-5k54 krb5kdc[533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:29.946678 548 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:29.948307 548 webserver.cc:460] Webserver started at http://127.0.0.1:34103/ using document root <none> and password file <none>
I20260807 22:55:29.949007 548 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:29.949155 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:29.949472 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:29.952561 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "3bd85c492c974e5da2ad09e6fec0eb8a"
format_stamp: "Formatted at 2026-08-07 22:55:29 on dist-test-slave-5k54"
server_key: "7e4e4000983375645260898123ee60e7"
server_key_iv: "aa7fad9bdde1b916ed80766a733e4e9b"
server_key_version: "encryptionkey@0"
I20260807 22:55:29.953361 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "3bd85c492c974e5da2ad09e6fec0eb8a"
format_stamp: "Formatted at 2026-08-07 22:55:29 on dist-test-slave-5k54"
server_key: "7e4e4000983375645260898123ee60e7"
server_key_iv: "aa7fad9bdde1b916ed80766a733e4e9b"
server_key_version: "encryptionkey@0"
I20260807 22:55:29.958663 548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:29.962185 564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:29.963676 548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:55:29.963866 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "3bd85c492c974e5da2ad09e6fec0eb8a"
format_stamp: "Formatted at 2026-08-07 22:55:29 on dist-test-slave-5k54"
server_key: "7e4e4000983375645260898123ee60e7"
server_key_iv: "aa7fad9bdde1b916ed80766a733e4e9b"
server_key_version: "encryptionkey@0"
I20260807 22:55:29.964072 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:29.987655 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:29.991837 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:29.992163 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:30.012624 548 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44611
I20260807 22:55:30.012662 621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44611 every 8 connection(s)
I20260807 22:55:30.014091 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:30.017886 622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:30.018415 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 548
I20260807 22:55:30.018610 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:30.019177 9995 external_mini_cluster.cc:1428] Setting key 54646a2ab2195f4e784aa3ab09c44acd
I20260807 22:55:30.032105 622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a: Bootstrap starting.
Aug 07 22:55:30 dist-test-slave-5k54 krb5kdc[533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:55:30.035910 622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:30.037129 622 log.cc:824] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:30.040027 622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a: No bootstrap required, opened a new log
I20260807 22:55:30.044656 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:30.021121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:54133 (local address 127.0.0.1:44611)
0807 22:55:30.021956 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:30.021968 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:30.022007 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:55:30.024319 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:30.024342 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:30.024805 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:30.025149 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:30.026881 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:30.028255 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:30.030095 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:30.030788 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:30.030927 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:30.036114 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:30.036124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:30.036139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:30.036182 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:30.039476 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:30.040309 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:30.040321 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:30.040331 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:30.040571 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:30.041148 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:30.041153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:30.041156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:30.041870 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:30.042272 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:30.042817 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 22:55:30.043191 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":397,"threads_started":1}
I20260807 22:55:30.048970 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:30.050104 622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260807 22:55:30.050487 622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:30.050614 622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd85c492c974e5da2ad09e6fec0eb8a, State: Initialized, Role: FOLLOWER
I20260807 22:55:30.051193 622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260807 22:55:30.051446 622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:30.051585 622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:30.051772 622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:30.054503 622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260807 22:55:30.055027 622 leader_election.cc:302] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd85c492c974e5da2ad09e6fec0eb8a; no voters:
I20260807 22:55:30.055725 622 leader_election.cc:288] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:30.055824 634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:30.058319 622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:30.058990 634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [term 1 LEADER]: Becoming Leader. State: Replica: 3bd85c492c974e5da2ad09e6fec0eb8a, State: Running, Role: LEADER
I20260807 22:55:30.059689 634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260807 22:55:30.070726 635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260807 22:55:30.070700 636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd85c492c974e5da2ad09e6fec0eb8a. Latest consensus state: current_term: 1 leader_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd85c492c974e5da2ad09e6fec0eb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260807 22:55:30.071172 635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:30.071182 636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:30.074172 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:30.053546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41835 (local address 127.0.0.1:44611)
0807 22:55:30.053738 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:30.053744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:30.054270 (+ 526us) server_negotiation.cc:407] Connection header received
0807 22:55:30.054386 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:30.054391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:30.054476 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:30.054595 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:30.056443 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:30.057343 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:30.058724 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:30.059417 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:30.059625 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:30.062950 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:30.062959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:30.062967 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:30.063008 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:30.069116 (+ 6108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:30.070169 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:30.070178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:30.070182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:30.070387 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:30.071049 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:30.071054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:30.071057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:30.071493 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:30.071755 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:30.073404 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0807 22:55:30.073640 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:55:30.083568 645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:30.089039 645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:30.098836 645 catalog_manager.cc:1347] Generated new cluster ID: 88545fdaad51414dbddaf8ec27e6e2c8
I20260807 22:55:30.098955 645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:30.133858 645 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:30.135211 645 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:30.143570 645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bd85c492c974e5da2ad09e6fec0eb8a: Generated new TSK 0
I20260807 22:55:30.144480 645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:30.162662 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 548
I20260807 22:55:30.164016 617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:30.259178 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 548
W20260807 22:55:30.280134 531 connection.cc:535] client connection to 127.0.0.1:44611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44611): Connection reset by peer (error 104)
2026-08-07T22:55:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:30 dist-test-slave-5k54 krb5kdc[656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:30 dist-test-slave-5k54 krb5kdc[656](info): set up 2 sockets
Aug 07 22:55:30 dist-test-slave-5k54 krb5kdc[656](info): commencing operation
krb5kdc: starting...
W20260807 22:55:31.432181 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:31 dist-test-slave-5k54 krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:31.751210 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:32.085654 669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:32.086041 669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:32.086205 669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:32.093556 669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:32.093724 669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:32.093842 669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:32.093947 669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:32.094031 669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:32.103976 669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44925
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:32.105381 669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:32.106760 669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:32.118937 675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:32.118997 678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:32.119469 676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:32.119680 669 server_base.cc:1034] running on GCE node
I20260807 22:55:32.120478 669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:32.121888 669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:32.123147 669 hybrid_clock.cc:648] HybridClock initialized: now 1786143332123095 us; error 43 us; skew 500 ppm
I20260807 22:55:32.123570 669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:32 dist-test-slave-5k54 krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:32.131713 669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:32.133248 669 webserver.cc:460] Webserver started at http://127.0.0.1:37091/ using document root <none> and password file <none>
I20260807 22:55:32.133970 669 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:32.134127 669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:32.134450 669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:32.137619 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a"
format_stamp: "Formatted at 2026-08-07 22:55:32 on dist-test-slave-5k54"
server_key: "f93e5961f4a19d18d6b241deef5103cf"
server_key_iv: "ab6ec0b1864cad177b66e5d12a65b1e3"
server_key_version: "encryptionkey@0"
I20260807 22:55:32.138417 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a"
format_stamp: "Formatted at 2026-08-07 22:55:32 on dist-test-slave-5k54"
server_key: "f93e5961f4a19d18d6b241deef5103cf"
server_key_iv: "ab6ec0b1864cad177b66e5d12a65b1e3"
server_key_version: "encryptionkey@0"
I20260807 22:55:32.143884 669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:55:32.147460 685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:32.148864 669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:32.149062 669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a"
format_stamp: "Formatted at 2026-08-07 22:55:32 on dist-test-slave-5k54"
server_key: "f93e5961f4a19d18d6b241deef5103cf"
server_key_iv: "ab6ec0b1864cad177b66e5d12a65b1e3"
server_key_version: "encryptionkey@0"
I20260807 22:55:32.149263 669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:32.172549 669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:32.176746 669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:32.177044 669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:32.196285 669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44925
I20260807 22:55:32.196327 740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44925 every 8 connection(s)
I20260807 22:55:32.197698 669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:32.200752 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 669
I20260807 22:55:32.200923 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:32.201390 9995 external_mini_cluster.cc:1428] Setting key d314734bde8bb732fc986bf4c57b29e5
I20260807 22:55:32.201665 741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:32.216902 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:32.203255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:60609 (local address 127.0.0.1:44925)
0807 22:55:32.204080 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:32.204095 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:32.204145 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:55:32.206573 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:32.206600 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:32.207152 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:32.207613 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:32.208159 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:32.210222 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:32.211605 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:32.212714 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:32.212913 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:32.214082 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:32.214093 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:32.214114 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:32.214861 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:32.214976 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:32.215016 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:32.215170 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:32.215409 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 22:55:32.215594 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":598,"thread_start_us":365,"threads_started":1}
I20260807 22:55:32.217240 741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a: Bootstrap starting.
I20260807 22:55:32.220631 741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:32.220825 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:32.221710 741 log.cc:824] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:32.224726 741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a: No bootstrap required, opened a new log
I20260807 22:55:32.232124 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:32.225031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:33537 (local address 127.0.0.1:44925)
0807 22:55:32.225244 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:32.225252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:32.225777 (+ 525us) server_negotiation.cc:407] Connection header received
0807 22:55:32.225971 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:32.225975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:32.226035 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:32.226113 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:32.226726 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:32.227546 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:32.229524 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:32.229968 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:32.230066 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:32.231258 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:32.231262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:32.231265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:32.231282 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:32.231321 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:32.231337 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:32.231468 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:32.231698 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 22:55:32.231771 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:55:32.234776 741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } }
I20260807 22:55:32.235186 741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:32.235301 741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fabe27a424d45aa8c5d8eb507bdfa7a, State: Initialized, Role: FOLLOWER
I20260807 22:55:32.235914 741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } }
I20260807 22:55:32.236171 741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:32.236335 741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:32.236534 741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:32.238858 741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } }
I20260807 22:55:32.239470 741 leader_election.cc:302] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fabe27a424d45aa8c5d8eb507bdfa7a; no voters:
I20260807 22:55:32.240146 741 leader_election.cc:288] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:32.240234 751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:32.240978 751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [term 1 LEADER]: Becoming Leader. State: Replica: 3fabe27a424d45aa8c5d8eb507bdfa7a, State: Running, Role: LEADER
I20260807 22:55:32.241582 751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } }
I20260807 22:55:32.242367 741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:32.244755 753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fabe27a424d45aa8c5d8eb507bdfa7a. Latest consensus state: current_term: 1 leader_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } } }
I20260807 22:55:32.244794 752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabe27a424d45aa8c5d8eb507bdfa7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44925 } } }
I20260807 22:55:32.245252 753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:32.245258 752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:32.246981 756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:32.252182 756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:32.261042 756 catalog_manager.cc:1347] Generated new cluster ID: 0bbb39b571cd437abb34551c4d228eeb
I20260807 22:55:32.261164 756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:32.290896 756 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:32.292213 756 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:32.304021 756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fabe27a424d45aa8c5d8eb507bdfa7a: Generated new TSK 0
I20260807 22:55:32.304859 756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:32.311901 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 669
W20260807 22:55:32.313547 736 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:60609
W20260807 22:55:32.314358 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 669: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:32.314507 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 669
W20260807 22:55:32.335680 654 connection.cc:535] client connection to 127.0.0.1:44925 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44925): Connection reset by peer (error 104)
2026-08-07T22:55:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:32 dist-test-slave-5k54 krb5kdc[775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:32 dist-test-slave-5k54 krb5kdc[775](info): set up 2 sockets
Aug 07 22:55:32 dist-test-slave-5k54 krb5kdc[775](info): commencing operation
krb5kdc: starting...
W20260807 22:55:33.548983 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:33 dist-test-slave-5k54 krb5kdc[775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:55:33.862867 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:34.205523 788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:34.205803 788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:34.205956 788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:34.213101 788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:34.213209 788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:34.213313 788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:34.213418 788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:34.213475 788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:34.223168 788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:34.224496 788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:34.225833 788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:34.238024 793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:34.238131 794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:34.238072 796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:34.239359 788 server_base.cc:1034] running on GCE node
I20260807 22:55:34.240170 788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:34.241633 788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:34.242888 788 hybrid_clock.cc:648] HybridClock initialized: now 1786143334242860 us; error 107 us; skew 500 ppm
I20260807 22:55:34.243270 788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:34 dist-test-slave-5k54 krb5kdc[775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143334, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:34.246542 788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:55:34.248076 788 webserver.cc:460] Webserver started at http://127.0.0.1:44439/ using document root <none> and password file <none>
I20260807 22:55:34.248767 788 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:34.248934 788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:34.249256 788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:34.252339 788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "7e1abecd7a7644488b99aa4215500fd0"
format_stamp: "Formatted at 2026-08-07 22:55:34 on dist-test-slave-5k54"
server_key: "5eb774a6fb39cdbd04011498376a7f03"
server_key_iv: "85fb6b26047fa1f4f54bdbd144985b3f"
server_key_version: "encryptionkey@0"
I20260807 22:55:34.253139 788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "7e1abecd7a7644488b99aa4215500fd0"
format_stamp: "Formatted at 2026-08-07 22:55:34 on dist-test-slave-5k54"
server_key: "5eb774a6fb39cdbd04011498376a7f03"
server_key_iv: "85fb6b26047fa1f4f54bdbd144985b3f"
server_key_version: "encryptionkey@0"
I20260807 22:55:34.258267 788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:34.261722 804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:34.263145 788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 22:55:34.263334 788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "7e1abecd7a7644488b99aa4215500fd0"
format_stamp: "Formatted at 2026-08-07 22:55:34 on dist-test-slave-5k54"
server_key: "5eb774a6fb39cdbd04011498376a7f03"
server_key_iv: "85fb6b26047fa1f4f54bdbd144985b3f"
server_key_version: "encryptionkey@0"
I20260807 22:55:34.263577 788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:34.269196 788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:34.273269 788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:34.273591 788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:34.291658 788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34995
I20260807 22:55:34.291661 859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34995 every 8 connection(s)
I20260807 22:55:34.293035 788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:34.293442 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 788
I20260807 22:55:34.293612 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:34.294150 9995 external_mini_cluster.cc:1428] Setting key 749d5e8cd113e7972e2b3eb21d405529
I20260807 22:55:34.297348 860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:34.302690 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:34.296183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:35581 (local address 127.0.0.1:34995)
0807 22:55:34.297127 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:34.297143 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:34.297513 (+ 370us) server_negotiation.cc:407] Connection header received
0807 22:55:34.299281 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:34.299300 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:34.299686 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:34.300081 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:34.301121 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:34.301127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:34.301142 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:34.301158 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:34.301226 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:34.301244 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:34.301485 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:34.301625 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 22:55:34.301810 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":380,"threads_started":1}
I20260807 22:55:34.306448 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:34.308938 860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0: Bootstrap starting.
I20260807 22:55:34.312745 860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:34.313357 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:34.310816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:47375 (local address 127.0.0.1:34995)
0807 22:55:34.310965 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:34.310969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:34.311420 (+ 451us) server_negotiation.cc:407] Connection header received
0807 22:55:34.311642 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:34.311646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:34.311690 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:34.311788 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:34.312717 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:34.312721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:34.312728 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:34.312742 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:34.312778 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:34.312792 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:34.312907 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:34.313041 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 22:55:34.313107 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:55:34.314121 860 log.cc:824] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:34.316833 860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0: No bootstrap required, opened a new log
I20260807 22:55:34.324354 860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } }
I20260807 22:55:34.324704 860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:34.324776 860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e1abecd7a7644488b99aa4215500fd0, State: Initialized, Role: FOLLOWER
I20260807 22:55:34.325302 860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } }
I20260807 22:55:34.325505 860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:34.325603 860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:34.325723 860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:34.328215 860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } }
I20260807 22:55:34.328714 860 leader_election.cc:302] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e1abecd7a7644488b99aa4215500fd0; no voters:
I20260807 22:55:34.329391 860 leader_election.cc:288] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:34.329551 872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:34.331496 872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 7e1abecd7a7644488b99aa4215500fd0, State: Running, Role: LEADER
I20260807 22:55:34.331804 860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:34.332297 872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } }
I20260807 22:55:34.335996 874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e1abecd7a7644488b99aa4215500fd0. Latest consensus state: current_term: 1 leader_uuid: "7e1abecd7a7644488b99aa4215500fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } } }
I20260807 22:55:34.336031 873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e1abecd7a7644488b99aa4215500fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abecd7a7644488b99aa4215500fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34995 } } }
I20260807 22:55:34.336441 874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:34.336455 873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:34.337406 883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:34.343493 883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:34.352231 883 catalog_manager.cc:1347] Generated new cluster ID: 971188b8f6364e1fa2c5bcbf95bb3109
I20260807 22:55:34.352341 883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:34.394416 883 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:34.394881 883 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:34.404093 883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e1abecd7a7644488b99aa4215500fd0: Generated new TSK 0
I20260807 22:55:34.404908 883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:34.411662 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 788
W20260807 22:55:34.412750 855 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:35581
W20260807 22:55:34.413440 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 788: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:34.413573 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 788
2026-08-07T22:55:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:34 dist-test-slave-5k54 krb5kdc[894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:34 dist-test-slave-5k54 krb5kdc[894](info): set up 2 sockets
Aug 07 22:55:34 dist-test-slave-5k54 krb5kdc[894](info): commencing operation
krb5kdc: starting...
W20260807 22:55:35.572355 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:35 dist-test-slave-5k54 krb5kdc[894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:35.887269 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33397
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:38123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:36.229540 909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:36.229842 909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:36.229947 909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:36.237129 909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:36.237246 909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:36.237294 909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:36.237357 909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:36.237396 909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:36.247134 909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:38123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33397
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:36.248461 909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:36.249751 909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:36.261677 914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:36.261822 917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:36.261822 915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:36.262744 909 server_base.cc:1034] running on GCE node
I20260807 22:55:36.263592 909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:36.264953 909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:36.266193 909 hybrid_clock.cc:648] HybridClock initialized: now 1786143336266146 us; error 55 us; skew 500 ppm
I20260807 22:55:36.266561 909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:36 dist-test-slave-5k54 krb5kdc[894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143336, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:36.269685 909 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:36.271206 909 webserver.cc:460] Webserver started at http://127.9.194.254:41303/ using document root <none> and password file <none>
I20260807 22:55:36.271944 909 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:36.272094 909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:36.272400 909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:36.275401 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "7f6f41a64af049af863482b4cbc464c6"
format_stamp: "Formatted at 2026-08-07 22:55:36 on dist-test-slave-5k54"
server_key: "5e5a4efdd89118167764ee4c23147266"
server_key_iv: "759726aa8b6dfa9f51b5d0cc9fcae643"
server_key_version: "encryptionkey@0"
I20260807 22:55:36.276376 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "7f6f41a64af049af863482b4cbc464c6"
format_stamp: "Formatted at 2026-08-07 22:55:36 on dist-test-slave-5k54"
server_key: "5e5a4efdd89118167764ee4c23147266"
server_key_iv: "759726aa8b6dfa9f51b5d0cc9fcae643"
server_key_version: "encryptionkey@0"
I20260807 22:55:36.281530 909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:55:36.285009 925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:36.286312 909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:55:36.286504 909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "7f6f41a64af049af863482b4cbc464c6"
format_stamp: "Formatted at 2026-08-07 22:55:36 on dist-test-slave-5k54"
server_key: "5e5a4efdd89118167764ee4c23147266"
server_key_iv: "759726aa8b6dfa9f51b5d0cc9fcae643"
server_key_version: "encryptionkey@0"
I20260807 22:55:36.286701 909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:36.310657 909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:36.314592 909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:36.314885 909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:36.333688 909 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33397
I20260807 22:55:36.333714 985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33397 every 8 connection(s)
I20260807 22:55:36.335080 909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:36.337822 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 909
I20260807 22:55:36.338016 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:36.338581 9995 external_mini_cluster.cc:1428] Setting key 747064d7f2bb323c5d4ec466093e584c
I20260807 22:55:36.339661 986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:36.353144 986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6: Bootstrap starting.
Aug 07 22:55:36 dist-test-slave-5k54 krb5kdc[894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:36.356568 986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:36.357784 986 log.cc:824] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:36.360769 986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6: No bootstrap required, opened a new log
I20260807 22:55:36.363946 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:36.340433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55827 (local address 127.9.194.254:33397)
0807 22:55:36.341242 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:36.341253 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:36.344007 (+ 2754us) server_negotiation.cc:407] Connection header received
0807 22:55:36.345698 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:36.345720 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:36.346137 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:36.346560 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:36.348203 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:36.349497 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:36.350797 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:36.351560 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:36.351759 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:36.356275 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:36.356283 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:36.356297 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:36.356337 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:36.359517 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:36.360324 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:36.360330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:36.360337 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:36.360514 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:36.361058 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:36.361062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:36.361064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:36.361584 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:36.361939 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:36.362482 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 22:55:36.362810 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":368,"threads_started":1}
I20260807 22:55:36.367951 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:36.369500 986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } }
I20260807 22:55:36.369848 986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:36.369951 986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6f41a64af049af863482b4cbc464c6, State: Initialized, Role: FOLLOWER
I20260807 22:55:36.370508 986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } }
I20260807 22:55:36.370733 986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:36.370867 986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:36.371063 986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:36.373639 986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } }
I20260807 22:55:36.374223 986 leader_election.cc:302] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f6f41a64af049af863482b4cbc464c6; no voters:
I20260807 22:55:36.374877 986 leader_election.cc:288] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:36.375041 997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:36.375716 997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7f6f41a64af049af863482b4cbc464c6, State: Running, Role: LEADER
I20260807 22:55:36.376567 997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } }
I20260807 22:55:36.378530 997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f6f41a64af049af863482b4cbc464c6. Latest consensus state: current_term: 1 leader_uuid: "7f6f41a64af049af863482b4cbc464c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } } }
I20260807 22:55:36.378795 997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:36.379971 999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f6f41a64af049af863482b4cbc464c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6f41a64af049af863482b4cbc464c6" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33397 } } }
I20260807 22:55:36.380283 999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:36.382999 986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:36.384585 1001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:36.390763 1001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:36.395524 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:36.373079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36825 (local address 127.9.194.254:33397)
0807 22:55:36.373274 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:36.373283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:36.373861 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:55:36.374048 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:36.374053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:36.374126 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:36.374251 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:36.376451 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:36.377632 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:36.379562 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:36.380371 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:36.380513 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:36.384505 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:36.384511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:36.384516 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:36.384553 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:36.388525 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:36.389464 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:36.389473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:36.389477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:36.389693 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:36.390245 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:36.390250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:36.390254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:36.390697 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:36.394936 (+ 4239us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:36.395091 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:55:36.395218 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:55:36.404644 1001 catalog_manager.cc:1347] Generated new cluster ID: f15ae2f8b1c7449b9b91053ee4708256
I20260807 22:55:36.404762 1001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:36.421455 1001 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:36.422706 1001 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:36.433390 1001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f6f41a64af049af863482b4cbc464c6: Generated new TSK 0
I20260807 22:55:36.434232 1001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:36.441146 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 909
I20260807 22:55:36.442297 981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:36.533888 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 909
W20260807 22:55:36.554679 892 connection.cc:535] client connection to 127.9.194.254:33397 recv error: Network error: failed to read from TLS socket (remote: 127.9.194.254:33397): Connection reset by peer (error 104)
2026-08-07T22:55:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2119 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:36 dist-test-slave-5k54 krb5kdc[1019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:36 dist-test-slave-5k54 krb5kdc[1019](info): set up 2 sockets
Aug 07 22:55:36 dist-test-slave-5k54 krb5kdc[1019](info): commencing operation
krb5kdc: starting...
W20260807 22:55:37.717882 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:37 dist-test-slave-5k54 krb5kdc[1019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:38.044560 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40193
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:40193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:38.422106 1034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:38.422396 1034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:38.422492 1034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:38.429697 1034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:38.429813 1034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:38.429870 1034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:38.429934 1034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:38.429982 1034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:38.439945 1034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:40193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:40193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:38.441315 1034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:38.442636 1034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:38.455120 1039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:38.455103 1043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:38.455202 1041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:38.456554 1034 server_base.cc:1034] running on GCE node
I20260807 22:55:38.457360 1034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:38.458916 1034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:38.460191 1034 hybrid_clock.cc:648] HybridClock initialized: now 1786143338460118 us; error 59 us; skew 500 ppm
I20260807 22:55:38.460600 1034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:38 dist-test-slave-5k54 krb5kdc[1019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143338, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:38.464128 1034 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:38.465746 1034 webserver.cc:460] Webserver started at http://127.9.194.254:39067/ using document root <none> and password file <none>
I20260807 22:55:38.466467 1034 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:38.466572 1034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:38.466918 1034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:38.470263 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "f893dfcd88ae4adb906700c265fb1aa5"
format_stamp: "Formatted at 2026-08-07 22:55:38 on dist-test-slave-5k54"
server_key: "8526ebe304a8931a5f7cc1d5bcc8ca00"
server_key_iv: "a9a78e57860c174b35423b7e1dbc2c34"
server_key_version: "encryptionkey@0"
I20260807 22:55:38.471160 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "f893dfcd88ae4adb906700c265fb1aa5"
format_stamp: "Formatted at 2026-08-07 22:55:38 on dist-test-slave-5k54"
server_key: "8526ebe304a8931a5f7cc1d5bcc8ca00"
server_key_iv: "a9a78e57860c174b35423b7e1dbc2c34"
server_key_version: "encryptionkey@0"
I20260807 22:55:38.476773 1034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 22:55:38.480557 1052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:38.482067 1034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:55:38.482209 1034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "f893dfcd88ae4adb906700c265fb1aa5"
format_stamp: "Formatted at 2026-08-07 22:55:38 on dist-test-slave-5k54"
server_key: "8526ebe304a8931a5f7cc1d5bcc8ca00"
server_key_iv: "a9a78e57860c174b35423b7e1dbc2c34"
server_key_version: "encryptionkey@0"
I20260807 22:55:38.482398 1034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:38.518193 1034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:38.523077 1034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:38.523332 1034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:38.542938 1034 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:40193
I20260807 22:55:38.542977 1108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:40193 every 8 connection(s)
I20260807 22:55:38.544396 1034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:38.548337 1109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:38.552851 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1034
I20260807 22:55:38.553071 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:38.553555 9995 external_mini_cluster.cc:1428] Setting key af0cc1c92e82b9307556ebff96e2e02a
I20260807 22:55:38.561314 1109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5: Bootstrap starting.
I20260807 22:55:38.565196 1109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:38.566396 1109 log.cc:824] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:38.569487 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:38.555469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38987 (local address 127.9.194.254:40193)
0807 22:55:38.556303 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:38.556327 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:38.556382 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:55:38.559225 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:38.559256 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:38.559808 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:38.560293 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:38.560815 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:38.562851 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:38.564285 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:38.565351 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:38.565545 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:38.566680 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:38.566690 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:38.566708 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:38.567497 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:38.567576 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:38.567609 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:38.567724 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:38.568052 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 22:55:38.568216 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":340,"threads_started":1}
I20260807 22:55:38.569805 1109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5: No bootstrap required, opened a new log
I20260807 22:55:38.573530 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:38.578437 1109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } }
I20260807 22:55:38.578961 1109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:38.579097 1109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f893dfcd88ae4adb906700c265fb1aa5, State: Initialized, Role: FOLLOWER
I20260807 22:55:38.579780 1109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } }
I20260807 22:55:38.580037 1109 raft_consensus.cc:388] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:38.580180 1109 raft_consensus.cc:482] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:38.580369 1109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:38.582791 1109 raft_consensus.cc:504] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } }
I20260807 22:55:38.583343 1109 leader_election.cc:302] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f893dfcd88ae4adb906700c265fb1aa5; no voters:
I20260807 22:55:38.584041 1109 leader_election.cc:288] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:38.584141 1120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:38.585807 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:38.578761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56623 (local address 127.9.194.254:40193)
0807 22:55:38.578949 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:38.578957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:38.579324 (+ 367us) server_negotiation.cc:407] Connection header received
0807 22:55:38.579551 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:38.579555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:38.579627 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:38.579737 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:38.580332 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:38.581148 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:38.582625 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:38.583234 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:38.583455 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:38.584804 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:38.584810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:38.584814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:38.584835 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:38.584881 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:38.584903 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:38.585006 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:38.585472 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 22:55:38.585521 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:55:38.586534 1120 raft_consensus.cc:686] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [term 1 LEADER]: Becoming Leader. State: Replica: f893dfcd88ae4adb906700c265fb1aa5, State: Running, Role: LEADER
I20260807 22:55:38.586625 1109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:38.587145 1120 consensus_queue.cc:237] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } }
I20260807 22:55:38.590734 1122 sys_catalog.cc:455] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f893dfcd88ae4adb906700c265fb1aa5. Latest consensus state: current_term: 1 leader_uuid: "f893dfcd88ae4adb906700c265fb1aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } } }
I20260807 22:55:38.590829 1121 sys_catalog.cc:455] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f893dfcd88ae4adb906700c265fb1aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f893dfcd88ae4adb906700c265fb1aa5" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 40193 } } }
I20260807 22:55:38.591248 1121 sys_catalog.cc:458] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:38.591128 1122 sys_catalog.cc:458] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:38.595082 1138 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:55:38.595337 1131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:38.600102 1131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:38.608150 1131 catalog_manager.cc:1347] Generated new cluster ID: eb948e72f17e4516aee8091394d2cee1
I20260807 22:55:38.608276 1131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:38.630637 1131 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:38.632658 1131 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:38.640523 1131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f893dfcd88ae4adb906700c265fb1aa5: Generated new TSK 0
I20260807 22:55:38.641611 1131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:38.657720 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1034
W20260807 22:55:38.659474 1103 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38987
W20260807 22:55:38.660300 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:38.660467 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1034
W20260807 22:55:38.680840 1017 connection.cc:535] client connection to 127.9.194.254:40193 recv error: Network error: failed to read from TLS socket (remote: 127.9.194.254:40193): Connection reset by peer (error 104)
2026-08-07T22:55:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2126 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:38 dist-test-slave-5k54 krb5kdc[1143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:38 dist-test-slave-5k54 krb5kdc[1143](info): set up 2 sockets
Aug 07 22:55:38 dist-test-slave-5k54 krb5kdc[1143](info): commencing operation
krb5kdc: starting...
W20260807 22:55:39.832840 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:40 dist-test-slave-5k54 krb5kdc[1143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:40.154119 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:37699
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:40.501227 1156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:40.501570 1156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:40.501729 1156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:40.509603 1156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:40.509768 1156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:40.509873 1156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:40.509984 1156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:40.510057 1156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:40.520011 1156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:37699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:40.521322 1156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:40.522629 1156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:40.534789 1161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:40.534883 1162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:40.534971 1164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:40.535640 1156 server_base.cc:1034] running on GCE node
I20260807 22:55:40.536317 1156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:40.537665 1156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:40.538897 1156 hybrid_clock.cc:648] HybridClock initialized: now 1786143340538856 us; error 34 us; skew 500 ppm
I20260807 22:55:40.539239 1156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:40 dist-test-slave-5k54 krb5kdc[1143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143340, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:40.542344 1156 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:40.543960 1156 webserver.cc:460] Webserver started at http://127.9.194.254:40875/ using document root <none> and password file <none>
I20260807 22:55:40.544610 1156 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:40.544757 1156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:40.545070 1156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:40.548084 1156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "1287e36d4dac48e0a34c9d7cb3c79193"
format_stamp: "Formatted at 2026-08-07 22:55:40 on dist-test-slave-5k54"
server_key: "e6169a42c7deeb919758de566b0df28c"
server_key_iv: "315d98bd1678774b005b42926c128f5d"
server_key_version: "encryptionkey@0"
I20260807 22:55:40.548863 1156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "1287e36d4dac48e0a34c9d7cb3c79193"
format_stamp: "Formatted at 2026-08-07 22:55:40 on dist-test-slave-5k54"
server_key: "e6169a42c7deeb919758de566b0df28c"
server_key_iv: "315d98bd1678774b005b42926c128f5d"
server_key_version: "encryptionkey@0"
I20260807 22:55:40.553984 1156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:55:40.557444 1172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:40.558732 1156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:55:40.558943 1156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "1287e36d4dac48e0a34c9d7cb3c79193"
format_stamp: "Formatted at 2026-08-07 22:55:40 on dist-test-slave-5k54"
server_key: "e6169a42c7deeb919758de566b0df28c"
server_key_iv: "315d98bd1678774b005b42926c128f5d"
server_key_version: "encryptionkey@0"
I20260807 22:55:40.559139 1156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:40.565009 1156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:40.568821 1156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:40.569118 1156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:40.588302 1156 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:37699
I20260807 22:55:40.588338 1224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:37699 every 8 connection(s)
I20260807 22:55:40.589695 1156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:40.593555 1225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:40.593710 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1156
I20260807 22:55:40.593895 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:40.594455 9995 external_mini_cluster.cc:1428] Setting key cc3cb068edf4c1bbbd72f47c4127d8a6
I20260807 22:55:40.602974 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:40.596294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55523 (local address 127.9.194.254:37699)
0807 22:55:40.597229 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:40.597241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:40.597281 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:55:40.599730 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:40.599753 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:40.600103 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:40.600432 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:40.601493 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:40.601499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:40.601511 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:40.601525 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:40.601584 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:40.601603 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:40.601761 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:40.601882 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 22:55:40.602091 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":434,"threads_started":1}
I20260807 22:55:40.606566 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:40.606633 1225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193: Bootstrap starting.
I20260807 22:55:40.610330 1225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:40.611732 1225 log.cc:824] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:40.614589 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:40.612063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59691 (local address 127.9.194.254:37699)
0807 22:55:40.612254 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:40.612260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:40.612594 (+ 334us) server_negotiation.cc:407] Connection header received
0807 22:55:40.612735 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:40.612743 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:40.612810 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:40.612934 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:40.613927 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:40.613931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:40.613935 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:40.613948 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:40.613985 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:40.613999 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:40.614082 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:40.614249 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:55:40.614293 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:55:40.615447 1225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193: No bootstrap required, opened a new log
I20260807 22:55:40.622933 1225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } }
I20260807 22:55:40.623263 1225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:40.623348 1225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1287e36d4dac48e0a34c9d7cb3c79193, State: Initialized, Role: FOLLOWER
I20260807 22:55:40.623906 1225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } }
I20260807 22:55:40.624127 1225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:40.624223 1225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:40.624349 1225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:40.626631 1225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } }
I20260807 22:55:40.627075 1225 leader_election.cc:302] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1287e36d4dac48e0a34c9d7cb3c79193; no voters:
I20260807 22:55:40.627673 1225 leader_election.cc:288] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:40.627770 1236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:40.628496 1236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [term 1 LEADER]: Becoming Leader. State: Replica: 1287e36d4dac48e0a34c9d7cb3c79193, State: Running, Role: LEADER
I20260807 22:55:40.629117 1236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } }
I20260807 22:55:40.629855 1225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:40.632336 1238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1287e36d4dac48e0a34c9d7cb3c79193. Latest consensus state: current_term: 1 leader_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } } }
I20260807 22:55:40.632833 1238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:40.634016 1237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1287e36d4dac48e0a34c9d7cb3c79193" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 37699 } } }
I20260807 22:55:40.634351 1237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:40.634800 1242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:40.640259 1242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:40.648819 1242 catalog_manager.cc:1347] Generated new cluster ID: d67094344a9144f7bd71aad752497b97
I20260807 22:55:40.648944 1242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:40.676568 1242 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:40.677029 1242 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:40.686993 1242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1287e36d4dac48e0a34c9d7cb3c79193: Generated new TSK 0
I20260807 22:55:40.687860 1242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:40.702155 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1156
W20260807 22:55:40.703222 1220 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55523
W20260807 22:55:40.703975 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1156: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:40.704113 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1156
2026-08-07T22:55:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:40 dist-test-slave-5k54 krb5kdc[1259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:40 dist-test-slave-5k54 krb5kdc[1259](info): set up 2 sockets
Aug 07 22:55:40 dist-test-slave-5k54 krb5kdc[1259](info): commencing operation
krb5kdc: starting...
W20260807 22:55:41.869412 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:42.182315 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33575
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:33575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:42.522605 1274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:42.522921 1274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:42.523033 1274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:42.531325 1274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:42.531502 1274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:42.531569 1274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:42.531635 1274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:42.531674 1274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:42.541752 1274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:33575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:42.543056 1274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:42.544457 1274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:42.556342 1280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:42.556288 1284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:42.556386 1281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:42.557500 1274 server_base.cc:1034] running on GCE node
I20260807 22:55:42.558358 1274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:42.559798 1274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:42.561093 1274 hybrid_clock.cc:648] HybridClock initialized: now 1786143342561036 us; error 44 us; skew 500 ppm
I20260807 22:55:42.561456 1274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143342, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:42.564731 1274 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:42.566365 1274 webserver.cc:460] Webserver started at http://127.9.194.254:32785/ using document root <none> and password file <none>
I20260807 22:55:42.567062 1274 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:42.567211 1274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:42.567580 1274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:42.570576 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "6e11540f0c02463093aa3bc49acbe88b"
format_stamp: "Formatted at 2026-08-07 22:55:42 on dist-test-slave-5k54"
server_key: "83170c44b09c63754c7e599a78bf591c"
server_key_iv: "a3cdc336ff0ba02ed59cac231e015f36"
server_key_version: "encryptionkey@0"
I20260807 22:55:42.571365 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "6e11540f0c02463093aa3bc49acbe88b"
format_stamp: "Formatted at 2026-08-07 22:55:42 on dist-test-slave-5k54"
server_key: "83170c44b09c63754c7e599a78bf591c"
server_key_iv: "a3cdc336ff0ba02ed59cac231e015f36"
server_key_version: "encryptionkey@0"
I20260807 22:55:42.576486 1274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 22:55:42.579911 1291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:42.581175 1274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:55:42.581355 1274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "6e11540f0c02463093aa3bc49acbe88b"
format_stamp: "Formatted at 2026-08-07 22:55:42 on dist-test-slave-5k54"
server_key: "83170c44b09c63754c7e599a78bf591c"
server_key_iv: "a3cdc336ff0ba02ed59cac231e015f36"
server_key_version: "encryptionkey@0"
I20260807 22:55:42.581552 1274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:42.602306 1274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:42.606781 1274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:42.607090 1274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:42.626188 1274 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:33575
I20260807 22:55:42.626226 1345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:33575 every 8 connection(s)
I20260807 22:55:42.627575 1274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:42.631244 1346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:42.633004 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1274
I20260807 22:55:42.633193 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:42.633759 9995 external_mini_cluster.cc:1428] Setting key a93d266e9ab6495f665473b052957336
I20260807 22:55:42.645551 1346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: Bootstrap starting.
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:42.649364 1346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:42.650457 1346 log.cc:824] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:42.653338 1346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: No bootstrap required, opened a new log
I20260807 22:55:42.656764 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:42.635663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59401 (local address 127.9.194.254:33575)
0807 22:55:42.636501 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:42.636512 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:42.636552 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:55:42.638106 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:42.638130 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:42.638527 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:42.638854 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:42.640621 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:42.641927 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:42.643844 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:42.644505 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:42.644649 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:42.649595 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:42.649604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:42.649617 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:42.649653 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:42.652738 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:42.653462 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:42.653473 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:42.653483 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:42.653651 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:42.654039 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:42.654046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:42.654048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:42.654549 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:42.654864 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:42.655360 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 22:55:42.655776 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":399,"threads_started":1}
I20260807 22:55:42.660426 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:42.661425 1346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } }
I20260807 22:55:42.661790 1346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:42.661895 1346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e11540f0c02463093aa3bc49acbe88b, State: Initialized, Role: FOLLOWER
I20260807 22:55:42.662526 1346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } }
I20260807 22:55:42.662783 1346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:42.662962 1346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:42.663148 1346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:42.665892 1346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } }
I20260807 22:55:42.666391 1346 leader_election.cc:302] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e11540f0c02463093aa3bc49acbe88b; no voters:
I20260807 22:55:42.667050 1346 leader_election.cc:288] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:42.667136 1356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:42.669716 1346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:42.673880 1356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [term 1 LEADER]: Becoming Leader. State: Replica: 6e11540f0c02463093aa3bc49acbe88b, State: Running, Role: LEADER
I20260807 22:55:42.674675 1356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } }
I20260807 22:55:42.678514 1357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e11540f0c02463093aa3bc49acbe88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } } }
I20260807 22:55:42.678763 1359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e11540f0c02463093aa3bc49acbe88b. Latest consensus state: current_term: 1 leader_uuid: "6e11540f0c02463093aa3bc49acbe88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e11540f0c02463093aa3bc49acbe88b" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 33575 } } }
I20260807 22:55:42.679111 1359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:42.679116 1357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:42.681667 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:42.664906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37249 (local address 127.9.194.254:33575)
0807 22:55:42.665081 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:42.665090 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:42.665494 (+ 404us) server_negotiation.cc:407] Connection header received
0807 22:55:42.665674 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:42.665679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:42.665753 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:42.665851 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:42.667571 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:42.668570 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:42.669869 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:42.670334 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:42.670466 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:42.673171 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:42.673180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:42.673187 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:42.673237 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:42.677252 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:42.678155 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:42.678163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:42.678168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:42.678344 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:42.678839 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:42.678845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:42.678847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:42.679346 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:42.679651 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:42.680941 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0807 22:55:42.681163 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 22:55:42.685653 1372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:55:42.685784 1372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:55:42.685973 1373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:42.687741 1373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:42.695995 1373 catalog_manager.cc:1347] Generated new cluster ID: 00e032d514ca47e6ba1f029056f6aabb
I20260807 22:55:42.696115 1373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:42.709931 1373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:42.711190 1373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:42.718803 1373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e11540f0c02463093aa3bc49acbe88b: Generated new TSK 0
I20260807 22:55:42.719867 1373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:42.729256 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1274
I20260807 22:55:42.730386 1340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:42.821210 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1274
2026-08-07T22:55:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2117 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1380](info): set up 2 sockets
Aug 07 22:55:42 dist-test-slave-5k54 krb5kdc[1380](info): commencing operation
krb5kdc: starting...
W20260807 22:55:44.002060 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:44 dist-test-slave-5k54 krb5kdc[1380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:44.323057 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35275
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:35275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:44.666729 1394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:44.667269 1394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:44.667517 1394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:44.675181 1394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:44.675294 1394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:44.675416 1394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:44.675565 1394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:44.675652 1394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:44.685832 1394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:35275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:35275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:44.687258 1394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:44.688648 1394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:44.700459 1401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:44.700591 1399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:44.700593 1403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:44.701419 1394 server_base.cc:1034] running on GCE node
I20260807 22:55:44.702224 1394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:44.703665 1394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:44.704900 1394 hybrid_clock.cc:648] HybridClock initialized: now 1786143344704850 us; error 37 us; skew 500 ppm
I20260807 22:55:44.705261 1394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:44 dist-test-slave-5k54 krb5kdc[1380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143344, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:44.712225 1394 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:44.713740 1394 webserver.cc:460] Webserver started at http://127.9.194.254:46239/ using document root <none> and password file <none>
I20260807 22:55:44.714427 1394 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:44.714571 1394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:44.714885 1394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:44.717934 1394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "d9cbdb7bcca34fd3beac6b66c34e27de"
format_stamp: "Formatted at 2026-08-07 22:55:44 on dist-test-slave-5k54"
server_key: "089ae8bc8da750fcb2d5ebe604da5994"
server_key_iv: "1a25698def759b8ecc6144c780f2ea20"
server_key_version: "encryptionkey@0"
I20260807 22:55:44.718703 1394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "d9cbdb7bcca34fd3beac6b66c34e27de"
format_stamp: "Formatted at 2026-08-07 22:55:44 on dist-test-slave-5k54"
server_key: "089ae8bc8da750fcb2d5ebe604da5994"
server_key_iv: "1a25698def759b8ecc6144c780f2ea20"
server_key_version: "encryptionkey@0"
I20260807 22:55:44.728071 1394 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.002s
I20260807 22:55:44.731603 1411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:44.732923 1394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:44.733121 1394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "d9cbdb7bcca34fd3beac6b66c34e27de"
format_stamp: "Formatted at 2026-08-07 22:55:44 on dist-test-slave-5k54"
server_key: "089ae8bc8da750fcb2d5ebe604da5994"
server_key_iv: "1a25698def759b8ecc6144c780f2ea20"
server_key_version: "encryptionkey@0"
I20260807 22:55:44.733322 1394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:44.750252 1394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:44.754346 1394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:44.754660 1394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:44.773594 1394 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:35275
I20260807 22:55:44.773624 1467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:35275 every 8 connection(s)
I20260807 22:55:44.774986 1394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:44.778635 1468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:44.783334 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1394
I20260807 22:55:44.783541 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:44.784083 9995 external_mini_cluster.cc:1428] Setting key 22b0c296a78d7ad698ffc1cc2ef073be
I20260807 22:55:44.790194 1468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de: Bootstrap starting.
I20260807 22:55:44.793771 1468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:44.795068 1468 log.cc:824] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:44.798130 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:44.785844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55573 (local address 127.9.194.254:35275)
0807 22:55:44.786631 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:44.786642 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:44.786690 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:55:44.788719 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:44.788741 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:44.789116 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:44.789419 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:44.790026 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:44.791377 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:44.793301 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:44.794100 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:44.794253 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:44.795650 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:44.795669 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:44.795692 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:44.796493 (+ 801us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:44.796597 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:44.796622 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:44.796708 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:44.796933 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 22:55:44.797192 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":368,"threads_started":1}
I20260807 22:55:44.798214 1468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de: No bootstrap required, opened a new log
I20260807 22:55:44.801934 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:44.808483 1468 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } }
I20260807 22:55:44.808872 1468 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:44.809003 1468 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cbdb7bcca34fd3beac6b66c34e27de, State: Initialized, Role: FOLLOWER
I20260807 22:55:44.809643 1468 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } }
I20260807 22:55:44.809921 1468 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:44.810118 1468 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:44.810286 1468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:44.812994 1468 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } }
I20260807 22:55:44.813612 1468 leader_election.cc:302] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9cbdb7bcca34fd3beac6b66c34e27de; no voters:
I20260807 22:55:44.813947 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:44.807290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45383 (local address 127.9.194.254:35275)
0807 22:55:44.807503 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:44.807509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:44.807976 (+ 467us) server_negotiation.cc:407] Connection header received
0807 22:55:44.808123 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:44.808128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:44.808204 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:44.808311 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:44.808806 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:44.809880 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:44.811276 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:44.812044 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:44.812166 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:44.813187 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:44.813192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:44.813197 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:44.813216 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:44.813258 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:44.813279 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:44.813374 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:44.813491 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:55:44.813575 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:55:44.814716 1468 leader_election.cc:288] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:44.814831 1478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:44.816025 1478 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [term 1 LEADER]: Becoming Leader. State: Replica: d9cbdb7bcca34fd3beac6b66c34e27de, State: Running, Role: LEADER
I20260807 22:55:44.816568 1478 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } }
I20260807 22:55:44.816916 1468 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:44.819507 1480 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9cbdb7bcca34fd3beac6b66c34e27de. Latest consensus state: current_term: 1 leader_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } } }
I20260807 22:55:44.819883 1480 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:44.820436 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbdb7bcca34fd3beac6b66c34e27de" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 35275 } } }
I20260807 22:55:44.820875 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:44.820889 1486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:44.835389 1486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:44.844918 1486 catalog_manager.cc:1347] Generated new cluster ID: 8420d7f79b8345a38714e84261a1adf0
I20260807 22:55:44.845055 1486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:44.857933 1486 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:44.859848 1486 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:44.878557 1486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9cbdb7bcca34fd3beac6b66c34e27de: Generated new TSK 0
I20260807 22:55:44.879726 1486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:44.895020 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1394
W20260807 22:55:44.896356 1463 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55573
W20260807 22:55:44.897091 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1394: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:44.897241 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1394
2026-08-07T22:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:45 dist-test-slave-5k54 krb5kdc[1500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:45 dist-test-slave-5k54 krb5kdc[1500](info): set up 2 sockets
Aug 07 22:55:45 dist-test-slave-5k54 krb5kdc[1500](info): commencing operation
krb5kdc: starting...
W20260807 22:55:46.056733 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:46 dist-test-slave-5k54 krb5kdc[1500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:46.379307 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38963
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:41751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:46.721926 1513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:46.722213 1513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:46.722410 1513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:46.729656 1513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:46.729765 1513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:46.729897 1513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:46.730003 1513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:46.730072 1513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:46.739899 1513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:41751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:38963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:46.741206 1513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:46.742530 1513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:46.754674 1519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:46.754753 1518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:46.754781 1521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:46.755779 1513 server_base.cc:1034] running on GCE node
I20260807 22:55:46.756532 1513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:46.757860 1513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:46.759075 1513 hybrid_clock.cc:648] HybridClock initialized: now 1786143346759022 us; error 49 us; skew 500 ppm
I20260807 22:55:46.759488 1513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:46 dist-test-slave-5k54 krb5kdc[1500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143346, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:46.762688 1513 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:46.764225 1513 webserver.cc:460] Webserver started at http://127.9.194.254:45403/ using document root <none> and password file <none>
I20260807 22:55:46.764901 1513 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:46.765014 1513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:46.765358 1513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:46.768400 1513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "b80d54e4f95144119721695873bab6e4"
format_stamp: "Formatted at 2026-08-07 22:55:46 on dist-test-slave-5k54"
server_key: "f0f0697fee77e9fee50323f6c7e24129"
server_key_iv: "c5fd607f938f6e0e6d768ac04f1db43f"
server_key_version: "encryptionkey@0"
I20260807 22:55:46.769229 1513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "b80d54e4f95144119721695873bab6e4"
format_stamp: "Formatted at 2026-08-07 22:55:46 on dist-test-slave-5k54"
server_key: "f0f0697fee77e9fee50323f6c7e24129"
server_key_iv: "c5fd607f938f6e0e6d768ac04f1db43f"
server_key_version: "encryptionkey@0"
I20260807 22:55:46.774493 1513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:46.777997 1528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:46.779356 1513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:55:46.779561 1513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "b80d54e4f95144119721695873bab6e4"
format_stamp: "Formatted at 2026-08-07 22:55:46 on dist-test-slave-5k54"
server_key: "f0f0697fee77e9fee50323f6c7e24129"
server_key_iv: "c5fd607f938f6e0e6d768ac04f1db43f"
server_key_version: "encryptionkey@0"
I20260807 22:55:46.779763 1513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:46.785434 1513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:46.789541 1513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:46.789831 1513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:46.808360 1513 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:38963
I20260807 22:55:46.808398 1586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:38963 every 8 connection(s)
I20260807 22:55:46.809799 1513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:46.813547 1587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:46.819985 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1513
I20260807 22:55:46.820184 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:46.820600 9995 external_mini_cluster.cc:1428] Setting key dada4355c45dc3d4cf2909dcedc86b03
I20260807 22:55:46.825317 1587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4: Bootstrap starting.
I20260807 22:55:46.829483 1587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:46.829902 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:46.822653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49217 (local address 127.9.194.254:38963)
0807 22:55:46.823554 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:46.823570 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:46.823623 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:55:46.825960 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:46.825991 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:46.826432 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:46.826905 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:46.827988 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:46.827996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:46.828013 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:46.828029 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:46.828101 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:46.828126 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:46.828344 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:46.828480 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 22:55:46.828676 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":376,"threads_started":1}
I20260807 22:55:46.830721 1587 log.cc:824] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:46.833853 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:46.834074 1587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4: No bootstrap required, opened a new log
I20260807 22:55:46.840853 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:46.838368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54843 (local address 127.9.194.254:38963)
0807 22:55:46.838558 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:46.838564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:46.838878 (+ 314us) server_negotiation.cc:407] Connection header received
0807 22:55:46.839020 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:46.839026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:46.839094 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:46.839215 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:46.840187 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:46.840193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:46.840197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:55:46.840214 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:46.840260 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:55:46.840287 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:46.840423 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:46.840492 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 22:55:46.840545 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:55:46.842875 1587 raft_consensus.cc:348] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } }
I20260807 22:55:46.843268 1587 raft_consensus.cc:374] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:55:46.843391 1587 raft_consensus.cc:729] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80d54e4f95144119721695873bab6e4, State: Initialized, Role: FOLLOWER
I20260807 22:55:46.844120 1587 consensus_queue.cc:260] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } }
I20260807 22:55:46.844449 1587 raft_consensus.cc:388] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:55:46.844609 1587 raft_consensus.cc:482] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:55:46.844817 1587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:46.846992 1587 raft_consensus.cc:504] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } }
I20260807 22:55:46.847596 1587 leader_election.cc:302] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b80d54e4f95144119721695873bab6e4; no voters:
I20260807 22:55:46.848176 1587 leader_election.cc:288] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:55:46.848321 1599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:46.849385 1599 raft_consensus.cc:686] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [term 1 LEADER]: Becoming Leader. State: Replica: b80d54e4f95144119721695873bab6e4, State: Running, Role: LEADER
I20260807 22:55:46.849944 1599 consensus_queue.cc:237] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } }
I20260807 22:55:46.850337 1587 sys_catalog.cc:564] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:46.852978 1600 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b80d54e4f95144119721695873bab6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } } }
I20260807 22:55:46.853194 1601 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b80d54e4f95144119721695873bab6e4. Latest consensus state: current_term: 1 leader_uuid: "b80d54e4f95144119721695873bab6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80d54e4f95144119721695873bab6e4" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 38963 } } }
I20260807 22:55:46.853502 1601 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:46.853498 1600 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:46.856199 1607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:46.861573 1607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:46.871701 1607 catalog_manager.cc:1347] Generated new cluster ID: 0df4b8322523439a8b452fa4f7559d37
I20260807 22:55:46.871869 1607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:46.886722 1607 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:46.887212 1607 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:46.897915 1607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b80d54e4f95144119721695873bab6e4: Generated new TSK 0
I20260807 22:55:46.898790 1607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:46.914165 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1513
W20260807 22:55:46.915216 1579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49217
W20260807 22:55:46.915926 9995 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1513: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:55:46.916070 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1513
2026-08-07T22:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2018 ms)
[----------] 18 tests from AuthTokenIssuingTest (37724 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:47 dist-test-slave-5k54 krb5kdc[1622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:47 dist-test-slave-5k54 krb5kdc[1622](info): set up 2 sockets
Aug 07 22:55:47 dist-test-slave-5k54 krb5kdc[1622](info): commencing operation
krb5kdc: starting...
W20260807 22:55:48.084945 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:48 dist-test-slave-5k54 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:48.419272 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:46185
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:48.761550 1635 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:48.761935 1635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:48.762063 1635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:48.762202 1635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:48.769469 1635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:48.769598 1635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:48.769712 1635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:48.769798 1635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:48.769893 1635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:48.779753 1635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:46185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:48.781091 1635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:48.782469 1635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:48.794520 1640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:48.794493 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:48.794754 1641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:48.795488 1635 server_base.cc:1034] running on GCE node
I20260807 22:55:48.796195 1635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:48.797523 1635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:48.798743 1635 hybrid_clock.cc:648] HybridClock initialized: now 1786143348798663 us; error 57 us; skew 500 ppm
I20260807 22:55:48.799058 1635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:48 dist-test-slave-5k54 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:48.802466 1635 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:48.804061 1635 webserver.cc:460] Webserver started at http://127.9.194.254:40213/ using document root <none> and password file <none>
I20260807 22:55:48.804796 1635 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:48.804890 1635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:48.805227 1635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:48.808269 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "da1bdd0a73be4a68998aa54c7db1e538"
format_stamp: "Formatted at 2026-08-07 22:55:48 on dist-test-slave-5k54"
server_key: "3f82fdc4662994023bda37e6853ab38b"
server_key_iv: "6ac0ae3b3afb0ff126f0ea3f3acb0073"
server_key_version: "encryptionkey@0"
I20260807 22:55:48.809060 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "da1bdd0a73be4a68998aa54c7db1e538"
format_stamp: "Formatted at 2026-08-07 22:55:48 on dist-test-slave-5k54"
server_key: "3f82fdc4662994023bda37e6853ab38b"
server_key_iv: "6ac0ae3b3afb0ff126f0ea3f3acb0073"
server_key_version: "encryptionkey@0"
I20260807 22:55:48.814174 1635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:55:48.817528 1650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:48.818831 1635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 22:55:48.819020 1635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "da1bdd0a73be4a68998aa54c7db1e538"
format_stamp: "Formatted at 2026-08-07 22:55:48 on dist-test-slave-5k54"
server_key: "3f82fdc4662994023bda37e6853ab38b"
server_key_iv: "6ac0ae3b3afb0ff126f0ea3f3acb0073"
server_key_version: "encryptionkey@0"
I20260807 22:55:48.819259 1635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:48.850446 1635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:48.855062 1635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:48.855396 1635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:48.875765 1635 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:46185
I20260807 22:55:48.875793 1706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:46185 every 8 connection(s)
I20260807 22:55:48.877444 1635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:48.881062 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1635
I20260807 22:55:48.881301 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:48.882201 9995 external_mini_cluster.cc:1428] Setting key 15a8d7ee4c03be2811f01dccaf1099a1
I20260807 22:55:48.882423 1707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:48.892558 1707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:48 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.194.253@KRBTEST.COM; defaulting to no policy
I20260807 22:55:48.916077 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.896646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.9.194.254:46185)
0807 22:55:48.899949 (+ 3303us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:48.899956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:48.900006 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:55:48.900198 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:48.900204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:48.900448 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:48.900656 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:48.901984 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:48.903150 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:48.904727 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:48.905284 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:48.905493 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:48.908838 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:48.908846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:48.908857 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:48.908885 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:48.912123 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:48.912824 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:48.912839 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:48.912841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:48.912987 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:48.913381 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:48.913388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:48.913395 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:48.913872 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:48.914089 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:48.914723 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 22:55:48.915079 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":3090,"spinlock_wait_cycles":23424,"thread_start_us":2855,"threads_started":1}
I20260807 22:55:48.916121 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.896646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:46185 (local address 127.0.0.1:41056)
0807 22:55:48.897605 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:48.897664 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:48.899802 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:48.900687 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:48.900693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:48.901431 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:48.901755 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:48.901772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:48.903303 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:48.903310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:48.904583 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:48.904595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:48.905524 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:48.905531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:48.905711 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:48.906421 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:48.906444 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:48.908616 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:48.912259 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:48.912272 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:48.912283 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:48.912683 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:48.913100 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:48.913106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:48.913110 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:48.913264 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:48.914008 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:48.914022 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:48.914255 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:55:48.914515 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 22:55:48.914915 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":407,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.253@KRBTEST.COM" created.
I20260807 22:55:48.919118 1707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:48.920328 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.919962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59656)
0807 22:55:48.920124 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:48.920226 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:55:48.920604 1654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
I20260807 22:55:48.922132 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.921799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59672)
0807 22:55:48.921947 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:48.922014 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:55:48.922346 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:35425: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
I20260807 22:55:48.958786 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:35425
--webserver_interface=127.9.194.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:55:48.971663 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } attempt: 1
I20260807 22:55:48.972992 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.972358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59682)
0807 22:55:48.972615 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:48.972756 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 22:55:48.974149 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:48.973673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59694)
0807 22:55:48.973864 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:48.973969 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 22:55:48.974382 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:35425: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
I20260807 22:55:49.042636 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } attempt: 2
I20260807 22:55:49.044056 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.043452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59708)
0807 22:55:49.043797 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.043906 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260807 22:55:49.045145 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.044760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59710)
0807 22:55:49.044933 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.045016 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 22:55:49.045338 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:35425: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
I20260807 22:55:49.136610 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } attempt: 3
I20260807 22:55:49.137821 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.137384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59726)
0807 22:55:49.137593 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.137688 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 22:55:49.138933 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.138545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59732)
0807 22:55:49.138724 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.138793 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 22:55:49.139119 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:35425: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
I20260807 22:55:49.282409 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } attempt: 4
I20260807 22:55:49.283767 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.283120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59738)
0807 22:55:49.283529 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.283623 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260807 22:55:49.284878 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.284469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59754)
0807 22:55:49.284654 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.284731 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 22:55:49.285066 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:35425: Network error: Client connection negotiation failed: client connection to 127.9.194.253:35425: connect: Connection refused (error 111)
W20260807 22:55:49.306679 1715 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:49.307071 1715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:49.307201 1715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:49.307348 1715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:49.314803 1715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:49.314942 1715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:49.315052 1715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:49.315152 1715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:49.315244 1715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:49.325716 1715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.194.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:49.327122 1715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:49.328564 1715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:49.341081 1720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:49.341027 1721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:49.341118 1723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:49.341740 1715 server_base.cc:1034] running on GCE node
I20260807 22:55:49.342540 1715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:49.343995 1715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:49.345213 1715 hybrid_clock.cc:648] HybridClock initialized: now 1786143349345155 us; error 32 us; skew 500 ppm
I20260807 22:55:49.345589 1715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:49.348999 1715 init.cc:377] Logged in from keytab as kudu/127.9.194.253@KRBTEST.COM (short username kudu)
I20260807 22:55:49.350566 1715 webserver.cc:460] Webserver started at http://127.9.194.253:38461/ using document root <none> and password file <none>
I20260807 22:55:49.351267 1715 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:49.351408 1715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:49.351776 1715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:49.354707 1715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data/instance:
uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "608c6e5c91d058ffe20f938675a4ff4e"
server_key_iv: "0065db631fc56973e553f7c9b1069915"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.355597 1715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal/instance:
uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "608c6e5c91d058ffe20f938675a4ff4e"
server_key_iv: "0065db631fc56973e553f7c9b1069915"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.361093 1715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:55:49.364629 1730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:49.366007 1715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:55:49.366202 1715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "608c6e5c91d058ffe20f938675a4ff4e"
server_key_iv: "0065db631fc56973e553f7c9b1069915"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.366433 1715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:49.397831 1715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:49.402253 1715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:49.402544 1715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:49.422191 1715 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.253:35425
I20260807 22:55:49.422212 1786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.253:35425 every 8 connection(s)
I20260807 22:55:49.423919 1715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
I20260807 22:55:49.428098 1788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:49.431155 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1715
I20260807 22:55:49.431376 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-1/wal/instance
I20260807 22:55:49.431964 9995 external_mini_cluster.cc:1428] Setting key 4aa64476bbfa72d5c825b9ac5f8ed564
I20260807 22:55:49.437883 1788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:49.464080 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.441611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.9.194.254:46185)
0807 22:55:49.442179 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:49.442186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:49.443334 (+ 1148us) server_negotiation.cc:407] Connection header received
0807 22:55:49.446367 (+ 3033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:49.446372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:49.446457 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:49.446557 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:49.448398 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.449653 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.451301 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.452090 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.452239 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.456088 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:49.456097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:49.456104 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:49.456166 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:49.460177 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.461089 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.461097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.461104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.461265 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.461715 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.461723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.461729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.462159 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:49.462333 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:49.463237 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 22:55:49.463550 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
I20260807 22:55:49.464759 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.442259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:46185 (local address 127.0.0.1:41066)
0807 22:55:49.443125 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.443183 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:49.446037 (+ 2854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:49.446846 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:49.446860 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:49.447543 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:49.448170 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.448187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.449854 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.449858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.451143 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.451155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.452318 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.452324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.452527 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.453261 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:49.453285 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:49.455833 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:49.460356 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.460364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.460378 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.460937 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.461385 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.461388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.461390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.461588 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.462347 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:49.462358 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:49.462745 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 22:55:49.463056 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 22:55:49.463568 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":320,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.252@KRBTEST.COM; defaulting to no policy
I20260807 22:55:49.466732 1788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.252@KRBTEST.COM" created.
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:49.481472 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.467610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59762)
0807 22:55:49.467876 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.467892 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:49.468075 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:49.468784 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:49.468788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:49.469237 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:49.469378 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.469386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.471115 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.471125 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.472078 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.472089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.472899 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.472904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.473105 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.473700 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:49.473718 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:49.475759 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:49.478876 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.478882 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.478889 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.479253 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.479675 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.479679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.479685 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.479833 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.480410 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:49.480415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:49.480560 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 22:55:49.480712 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 22:55:49.481010 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:55:49.481583 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.467744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.9.194.253:35425)
0807 22:55:49.468383 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:49.468389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:49.468413 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:55:49.468461 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:49.468467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:49.468640 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:49.468802 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:49.469554 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.470931 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.472209 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.472753 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.472891 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.475988 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:49.475999 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:49.476012 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:49.476040 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:49.478748 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.479392 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.479398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.479400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.479568 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.479942 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.479947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.479950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.480289 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:49.480521 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:49.480866 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 22:55:49.481129 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":503,"thread_start_us":274,"threads_started":1}
I20260807 22:55:49.484079 1788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:49.485368 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.484849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:39978)
0807 22:55:49.485059 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.485203 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 22:55:49.485673 1731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.487385 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.486918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:39990)
0807 22:55:49.487118 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.487238 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 22:55:49.487700 1788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
I20260807 22:55:49.510761 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:45501
--webserver_interface=127.9.194.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:55:49.537146 1788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 1
I20260807 22:55:49.538506 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.537882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40004)
0807 22:55:49.538131 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.538276 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 22:55:49.539688 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.539184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40018)
0807 22:55:49.539394 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.539527 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:55:49.539965 1788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.584324 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } attempt: 5
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:49.600142 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.585452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.9.194.253:35425)
0807 22:55:49.585670 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:49.585677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:49.585700 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:55:49.585787 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:49.585792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:49.585875 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:49.585989 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:49.586791 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.587982 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.589233 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.590002 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.590140 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.593989 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:49.593994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:49.593998 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:49.594035 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:49.597037 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.597978 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.597983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.597985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.598114 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:49.598600 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:49.598604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:49.598606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:49.598962 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:49.599083 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:49.599672 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 22:55:49.599911 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:55:49.600296 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.585179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59770)
0807 22:55:49.585426 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.585449 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:49.585595 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:49.586044 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:49.586048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:49.586448 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:49.586625 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.586635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.588162 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.588167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.589064 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.589077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.590163 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.590168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:49.590345 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:49.591066 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:49.591086 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:49.593693 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:49.597216 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.597225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.597232 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.597851 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.598233 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:49.598238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:49.598242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:49.598446 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:49.599092 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:49.599097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:49.599321 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 22:55:49.599550 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 22:55:49.599908 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 22:55:49.601624 1707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:49.602775 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.602293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40022)
0807 22:55:49.602499 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.602598 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 22:55:49.603991 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.603522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40024)
0807 22:55:49.603714 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.603820 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:55:49.604244 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.608211 1788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 2
I20260807 22:55:49.609292 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.608798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40036)
0807 22:55:49.609014 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.609116 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:55:49.610397 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.609974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40042)
0807 22:55:49.610159 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.610247 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 22:55:49.610632 1788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.655517 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 1
I20260807 22:55:49.656826 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.656242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40052)
0807 22:55:49.656482 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.656600 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 22:55:49.658097 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.657757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40058)
0807 22:55:49.657905 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.657985 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 22:55:49.658378 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.701895 1788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 3
I20260807 22:55:49.703233 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.702631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40074)
0807 22:55:49.702894 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.703015 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 22:55:49.704494 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.703952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40082)
0807 22:55:49.704197 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.704311 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260807 22:55:49.704749 1788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.726665 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 2
I20260807 22:55:49.728209 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.727669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40094)
0807 22:55:49.727927 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.728031 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 22:55:49.729343 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.728962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40108)
0807 22:55:49.729137 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.729209 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 22:55:49.729585 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.835839 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 3
I20260807 22:55:49.836956 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.836485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40112)
0807 22:55:49.836691 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.836787 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 22:55:49.837992 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.837616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40128)
0807 22:55:49.837821 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.837892 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 22:55:49.838188 1707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
I20260807 22:55:49.847994 1788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 4
I20260807 22:55:49.849215 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.848637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40134)
0807 22:55:49.848908 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.849021 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 22:55:49.850297 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.849933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40150)
0807 22:55:49.850095 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.850177 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 22:55:49.850528 1788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:45501: Network error: Client connection negotiation failed: client connection to 127.9.194.252:45501: connect: Connection refused (error 111)
W20260807 22:55:49.873009 1796 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:49.873425 1796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:49.873543 1796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:49.873672 1796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:49.881472 1796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:49.881567 1796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:49.881660 1796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:49.881753 1796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:49.881860 1796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:49.892062 1796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:46185,127.9.194.253:35425,127.9.194.252:45501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:45501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.194.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:49.893437 1796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:49.894793 1796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:49.907075 1803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:49.907047 1804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:49.907047 1806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:49.907940 1796 server_base.cc:1034] running on GCE node
I20260807 22:55:49.908751 1796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:49.910132 1796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:49.911381 1796 hybrid_clock.cc:648] HybridClock initialized: now 1786143349911309 us; error 50 us; skew 500 ppm
I20260807 22:55:49.911763 1796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:49.920608 1796 init.cc:377] Logged in from keytab as kudu/127.9.194.252@KRBTEST.COM (short username kudu)
I20260807 22:55:49.922201 1796 webserver.cc:460] Webserver started at http://127.9.194.252:45267/ using document root <none> and password file <none>
I20260807 22:55:49.922974 1796 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:49.923112 1796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:49.923502 1796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:49.926550 1796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data/instance:
uuid: "e59d2bbae56349b784bbc4e52c2e3ef7"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "c9657742ef456b2eade050f50708d335"
server_key_iv: "b6ff750105e5fbf0ba601b7c735863d4"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.927386 1796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal/instance:
uuid: "e59d2bbae56349b784bbc4e52c2e3ef7"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "c9657742ef456b2eade050f50708d335"
server_key_iv: "b6ff750105e5fbf0ba601b7c735863d4"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.932726 1796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 22:55:49.936180 1814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:49.937562 1796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:55:49.937702 1796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "e59d2bbae56349b784bbc4e52c2e3ef7"
format_stamp: "Formatted at 2026-08-07 22:55:49 on dist-test-slave-5k54"
server_key: "c9657742ef456b2eade050f50708d335"
server_key_iv: "b6ff750105e5fbf0ba601b7c735863d4"
server_key_version: "encryptionkey@0"
I20260807 22:55:49.937880 1796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:49.955237 1796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:49.959729 1796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:49.959952 1796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:49.978470 1796 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.252:45501
I20260807 22:55:49.978493 1872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.252:45501 every 8 connection(s)
I20260807 22:55:49.979832 1796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
I20260807 22:55:49.983543 1873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:49.985210 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 1796
I20260807 22:55:49.985394 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142983724342-9995-0/minicluster-data/master-2/wal/instance
I20260807 22:55:49.985857 9995 external_mini_cluster.cc:1428] Setting key e34f5d68c56f410487ca7adf2d22f91f
I20260807 22:55:49.993242 1873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:49 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:50.009938 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.987803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.9.194.252:45501)
0807 22:55:49.988663 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:49.988682 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:49.988733 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:55:49.990417 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:49.990452 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:49.990966 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:49.991389 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:49.992790 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.994749 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.996125 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.997298 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.997477 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.001636 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.001643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.001655 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.001691 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.005406 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.006161 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.006168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.006175 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.006327 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.006958 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.006965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.006969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.007673 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.007963 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.008401 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.008798 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":337,"threads_started":1}
I20260807 22:55:50.012194 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.996126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:46185 (local address 127.0.0.1:41078)
0807 22:55:49.997077 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:49.997112 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:49.997298 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:49.997681 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:49.997688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:49.998476 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:49.998733 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:49.998742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.999881 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:49.999887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.000994 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.001006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.001766 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.001773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.002097 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.002916 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:50.002934 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:50.005699 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:50.009516 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.009531 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.009538 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.009986 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.010466 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.010472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.010476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.010638 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.011209 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:50.011216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:50.011411 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 22:55:50.011641 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 22:55:50.011887 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":571,"threads_started":1}
I20260807 22:55:50.012415 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:49.996151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.9.194.254:46185)
0807 22:55:49.996770 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:49.996779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:49.997214 (+ 435us) server_negotiation.cc:407] Connection header received
0807 22:55:49.997440 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:49.997445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:49.997507 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:49.997616 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:49.998896 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:49.999733 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.001149 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.001554 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.001700 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.005967 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.005973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.005976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.006010 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.009369 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.010197 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.010202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.010204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.010337 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.010755 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.010760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.010764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.011047 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.011223 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.011775 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.012013 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":296,"threads_started":1}
I20260807 22:55:50.014528 1873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:50.015399 1707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 4
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:50.031772 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.015345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:35425 (local address 127.0.0.1:59772)
0807 22:55:50.015868 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:50.015886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:50.016048 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:50.016557 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:50.016561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:50.017031 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:50.017256 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.017270 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.018311 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.018317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.019664 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.019679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.020736 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.020741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.021001 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.021582 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:50.021607 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:50.025125 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:50.028818 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.028824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.028828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.029424 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.029871 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.029876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.029880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.030085 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.030730 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:50.030736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:50.030908 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 22:55:50.031058 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:55:50.031352 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":2}
I20260807 22:55:50.032279 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.015515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.9.194.253:35425)
0807 22:55:50.015702 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.015707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.015958 (+ 251us) server_negotiation.cc:407] Connection header received
0807 22:55:50.016197 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.016205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.016286 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.016361 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.017432 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.018160 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.019816 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.020368 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.020520 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.025331 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.025338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.025343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.025388 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.028680 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.029567 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.029573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.029577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.029744 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.030253 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.030258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.030261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.030601 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.030753 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.031195 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.031470 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:55:50.032505 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.016002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40162)
0807 22:55:50.016186 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:50.016203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:50.016299 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:50.016744 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:50.016749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:50.017244 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:50.017426 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.017438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.018752 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.018758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.020297 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.020311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.021047 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.021053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.021253 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.021797 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:50.021814 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:50.025125 (+ 3311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:50.029871 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.029876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.029880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.030379 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.030795 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.030801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.030804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.030994 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.031630 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:50.031636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:50.031793 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 22:55:50.031937 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:55:50.032160 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260807 22:55:50.032820 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.013679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.9.194.254:46185)
0807 22:55:50.013884 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.013889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.013906 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:50.014069 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.014074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.014166 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.014270 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.015934 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.017002 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.018839 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.019298 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.019463 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.024984 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.024989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.024992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.025022 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.028420 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.029266 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.029274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.029279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.029429 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.030161 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.030168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.030176 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.030617 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.030895 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.032185 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.032435 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":4992}
I20260807 22:55:50.033514 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.016285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.9.194.252:45501)
0807 22:55:50.016438 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.016444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.016459 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:55:50.016525 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.016531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.016591 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.016692 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.017568 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.018599 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.020454 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.020897 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.021039 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.026175 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.026180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.026184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.026219 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.029750 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.030512 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.030517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.030521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.030635 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.031118 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.031122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.031125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.031481 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.032763 (+ 1282us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.032916 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.033150 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":52}
I20260807 22:55:50.033480 1873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:50.039885 1707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: Bootstrap starting.
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
I20260807 22:55:50.044468 1707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:50.045922 1707 log.cc:824] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:50.049369 1707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: No bootstrap required, opened a new log
I20260807 22:55:50.050714 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.034568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.9.194.253:35425)
0807 22:55:50.034734 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.034739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.034880 (+ 141us) server_negotiation.cc:407] Connection header received
0807 22:55:50.035215 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.035220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.035294 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.035394 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.036638 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.037337 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.038526 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.038950 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.039098 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.044390 (+ 5292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.044396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.044399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.044454 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.047666 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.048530 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.048539 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.048543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.048710 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.049175 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.049180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.049183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.049580 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.049860 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.050120 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.050345 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:55:50.051595 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.034702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40168)
0807 22:55:50.035049 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:50.035066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:50.035152 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:50.035702 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:50.035706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:50.036158 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:50.036325 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.036335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.037679 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.037683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.038914 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.038930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.040186 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.040192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.040453 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.041127 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:50.041148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:50.044181 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:50.047987 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.047995 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.047999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.048503 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.048894 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.048901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.048904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.049048 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.050531 (+ 1483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:50.050535 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:50.050702 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 22:55:50.050848 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 22:55:50.051172 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260807 22:55:50.052420 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.035097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.9.194.252:45501)
0807 22:55:50.035281 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.035287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.035304 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:50.035350 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.035355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.035462 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.035556 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.036487 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.037533 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.039065 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.039724 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.039845 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.044417 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.044423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.044427 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.044465 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.047846 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.048625 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.048630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.048633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.048754 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.049931 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.049936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.049939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.050315 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.050468 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.051856 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.052078 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:55:50.055682 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:50.058073 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: Bootstrap starting.
I20260807 22:55:50.058344 1677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "da1bdd0a73be4a68998aa54c7db1e538"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41092
I20260807 22:55:50.060731 1707 raft_consensus.cc:348] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.061148 1707 raft_consensus.cc:729] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1bdd0a73be4a68998aa54c7db1e538, State: Initialized, Role: FOLLOWER
I20260807 22:55:50.061698 1707 consensus_queue.cc:260] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.062213 1873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:50.062790 1880 sys_catalog.cc:455] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.063215 1880 sys_catalog.cc:458] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.063632 1873 log.cc:824] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:50.064193 1707 sys_catalog.cc:564] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:50.064360 1677 raft_consensus.cc:482] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:55:50.064611 1677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:50.067489 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: No bootstrap required, opened a new log
I20260807 22:55:50.068497 1677 raft_consensus.cc:504] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.072463 1677 leader_election.cc:288] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [CANDIDATE]: Term 1 election: Requested vote from peers 94377cb76c4a4d5dae43e0b6acc7f3ac (127.9.194.253:35425), e59d2bbae56349b784bbc4e52c2e3ef7 (127.9.194.252:45501)
I20260807 22:55:50.079048 1873 raft_consensus.cc:348] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.079536 1873 raft_consensus.cc:729] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59d2bbae56349b784bbc4e52c2e3ef7, State: Initialized, Role: FOLLOWER
I20260807 22:55:50.080070 1873 consensus_queue.cc:260] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.074249 1635 master_runner.cc:186] Error getting master registration for 127.9.194.253:35425: OK, instance_id { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" instance_seqno: 1786143349416664 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:55:50.086874 1635 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:55:50.087050 1896 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.087386 1896 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.089051 1873 sys_catalog.cc:564] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:50.089119 1843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da1bdd0a73be4a68998aa54c7db1e538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7"
I20260807 22:55:50.089756 1843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:50.094712 1843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1bdd0a73be4a68998aa54c7db1e538 in term 1.
I20260807 22:55:50.095762 1651 leader_election.cc:302] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da1bdd0a73be4a68998aa54c7db1e538, e59d2bbae56349b784bbc4e52c2e3ef7; no voters:
I20260807 22:55:50.096419 1880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:55:50.099538 1880 raft_consensus.cc:686] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [term 1 LEADER]: Becoming Leader. State: Replica: da1bdd0a73be4a68998aa54c7db1e538, State: Running, Role: LEADER
I20260807 22:55:50.101153 1880 consensus_queue.cc:237] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
W20260807 22:55:50.101907 1893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:55:50.102036 1893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 22:55:50.108414 1656 tablet.cc:2369] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:50.116458 1905 sys_catalog.cc:455] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da1bdd0a73be4a68998aa54c7db1e538. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.117053 1905 sys_catalog.cc:458] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:50.118042 1910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:50.120443 1910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:55:50.144013 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.103162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.9.194.254:46185)
0807 22:55:50.103346 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.103352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.109895 (+ 6543us) server_negotiation.cc:407] Connection header received
0807 22:55:50.110136 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.110141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.110245 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.110345 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.113432 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.114871 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.121066 (+ 6195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.121838 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.121960 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.133034 (+ 11074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.133042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.133050 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.133124 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.137080 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.137997 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.138006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.138010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.138192 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.140733 (+ 2541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.140742 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.140748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.141269 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.142071 (+ 802us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.142439 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.143038 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:55:50.149715 1788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } attempt: 5
I20260807 22:55:50.155238 1843 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [term 1 FOLLOWER]: Refusing update from remote peer da1bdd0a73be4a68998aa54c7db1e538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:50.155325 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.124006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.9.194.252:45501)
0807 22:55:50.124190 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.124196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.127989 (+ 3793us) server_negotiation.cc:407] Connection header received
0807 22:55:50.131017 (+ 3028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.131024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.131121 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.131202 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.133166 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.134268 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.136246 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.136985 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.137093 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.142713 (+ 5620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.142721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.142726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.142768 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.148998 (+ 6230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.152757 (+ 3759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.152762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.152765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.152942 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.153413 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.153418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.153421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.153799 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.154097 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.154703 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.154970 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:55:50.156347 1880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:50.165123 1919 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Aug 07 22:55:50 dist-test-slave-5k54 krb5kdc[1622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
I20260807 22:55:50.171100 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.104390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.9.194.253:35425)
0807 22:55:50.104639 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.104648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.127321 (+ 22673us) server_negotiation.cc:407] Connection header received
0807 22:55:50.127701 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.127709 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.127801 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.127880 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.129720 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.130807 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.138743 (+ 7936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.139538 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.139651 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.144768 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.144777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.144804 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.144865 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.148759 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.149693 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.149700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.149703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.149914 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.168308 (+ 18394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.168315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.168320 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.168746 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.169028 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.170296 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.170616 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 22:55:50.171729 1796 master_runner.cc:186] Error getting master registration for 127.9.194.253:35425: OK, instance_id { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" instance_seqno: 1786143349416664 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:55:50.184811 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.150427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:45501 (local address 127.0.0.1:40186)
0807 22:55:50.150795 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:50.150815 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:50.150915 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:50.157509 (+ 6594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:50.157517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:50.158063 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:50.158308 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.158324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.161377 (+ 3053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.161383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.162775 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:50.162791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.165091 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.165099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:50.165420 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.166385 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:50.166445 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:50.171386 (+ 4941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:50.179208 (+ 7822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.179241 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.179246 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.181061 (+ 1815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.182195 (+ 1134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:50.182201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:50.182204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:50.182399 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:50.183736 (+ 1337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:50.183744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:50.183919 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 22:55:50.184075 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 22:55:50.184388 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260807 22:55:50.185034 1796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:55:50.186897 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.151123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.9.194.252:45501)
0807 22:55:50.156751 (+ 5628us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.156756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.156776 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:55:50.156822 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.156826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.156898 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.158495 (+ 1597us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.158634 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.160765 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.162989 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.163842 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.165840 (+ 1998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.171602 (+ 5762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:50.171611 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:50.171619 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:50.171667 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:50.178999 (+ 7332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.181342 (+ 2343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.181351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.181386 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.181704 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:50.182613 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:50.182619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:50.182654 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:50.183487 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:50.185191 (+ 1704us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.185524 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.186071 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5530,"thread_start_us":35149,"threads_started":1}
I20260807 22:55:50.194149 1788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: Bootstrap starting.
I20260807 22:55:50.199394 1788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:50.201165 1788 log.cc:824] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:50.205462 1896 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da1bdd0a73be4a68998aa54c7db1e538. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.205772 1896 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.206468 1788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: No bootstrap required, opened a new log
I20260807 22:55:50.207952 1913 mvcc.cc:205] Tried to move back new op lower bound from 7316043162219364352 to 7316043162020020224. Current Snapshot: MvccSnapshot[applied={T|T < 7316043162219364352}]
I20260807 22:55:50.216193 1896 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.216553 1896 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.231496 1910 catalog_manager.cc:1347] Generated new cluster ID: 1d32dc3d06ef48739b27f6478da1cd7a
I20260807 22:55:50.231637 1910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:50.232887 1880 sys_catalog.cc:455] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.233206 1880 sys_catalog.cc:458] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:50.234704 1788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.235101 1788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94377cb76c4a4d5dae43e0b6acc7f3ac, State: Initialized, Role: FOLLOWER
I20260807 22:55:50.235633 1788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } }
I20260807 22:55:50.246999 1925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.247593 1925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.250358 1788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:50.250420 1761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da1bdd0a73be4a68998aa54c7db1e538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac"
I20260807 22:55:50.251371 1910 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:50.251211 1759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:50.253955 1910 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:50.256867 1759 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [term 1 FOLLOWER]: Refusing update from remote peer da1bdd0a73be4a68998aa54c7db1e538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:50.262554 1761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1bdd0a73be4a68998aa54c7db1e538 in term 1.
I20260807 22:55:50.265367 1880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:50.272869 1910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538: Generated new TSK 0
I20260807 22:55:50.274415 1910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:55:50.313131 1933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.313591 1933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.314076 1925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader da1bdd0a73be4a68998aa54c7db1e538. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.314051 1905 sys_catalog.cc:455] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da1bdd0a73be4a68998aa54c7db1e538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da1bdd0a73be4a68998aa54c7db1e538" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 46185 } } peers { permanent_uuid: "94377cb76c4a4d5dae43e0b6acc7f3ac" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 35425 } } peers { permanent_uuid: "e59d2bbae56349b784bbc4e52c2e3ef7" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 45501 } } }
I20260807 22:55:50.314349 1925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:50.314414 1905 sys_catalog.cc:458] T 00000000000000000000000000000000 P da1bdd0a73be4a68998aa54c7db1e538 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:50.334559 1945 catalog_manager.cc:1259] Loaded cluster ID: 1d32dc3d06ef48739b27f6478da1cd7a
I20260807 22:55:50.334736 1945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: loading cluster ID for follower catalog manager: success
I20260807 22:55:50.338940 1945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: acquiring CA information for follower catalog manager: success
I20260807 22:55:50.341602 1945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 94377cb76c4a4d5dae43e0b6acc7f3ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 22:55:50.400633 1793 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.378768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.9.194.253:35425)
0807 22:55:50.379329 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.379338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.398523 (+ 19185us) server_negotiation.cc:407] Connection header received
0807 22:55:50.398709 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.398716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.398823 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.398948 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.400361 (+ 1413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59798: BlockingRecv error: recv got EOF from 127.0.0.1:59798 (error 108)
Metrics: {"server-negotiator.queue_time_us":370}
W20260807 22:55:50.401831 1877 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.378709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.9.194.254:46185)
0807 22:55:50.379091 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.379106 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.397753 (+ 18647us) server_negotiation.cc:407] Connection header received
0807 22:55:50.397990 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.397998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.398105 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.398224 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.401574 (+ 3350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41108: BlockingRecv error: recv got EOF from 127.0.0.1:41108 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
W20260807 22:55:50.403069 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.398996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.9.194.252:45501)
0807 22:55:50.399208 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.399214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.400493 (+ 1279us) server_negotiation.cc:407] Connection header received
0807 22:55:50.401712 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.401717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.401796 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.401872 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.402839 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40200: BlockingRecv error: recv got EOF from 127.0.0.1:40200 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:55:50.414961 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.408772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.9.194.253:35425)
0807 22:55:50.408983 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.408992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.409480 (+ 488us) server_negotiation.cc:407] Connection header received
0807 22:55:50.409733 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.409737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.409876 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.410044 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.410444 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.411346 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.412676 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.413128 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.413266 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.413774 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.414279 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.414450 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.414532 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.414623 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 22:55:50.442114 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.421860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.9.194.252:45501)
0807 22:55:50.422049 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.422055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.440134 (+ 18079us) server_negotiation.cc:407] Connection header received
0807 22:55:50.440419 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.440425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.440545 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.440643 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.441854 (+ 1211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40212: BlockingRecv error: recv got EOF from 127.0.0.1:40212 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:55:50.480804 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.468856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.9.194.253:35425)
0807 22:55:50.469153 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.469163 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.475342 (+ 6179us) server_negotiation.cc:407] Connection header received
0807 22:55:50.475661 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.475666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.475764 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.475822 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.476427 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.477309 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.478883 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.479369 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.479564 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.479968 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.480125 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.480229 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.480379 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.480468 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 22:55:50.490109 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.487603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.9.194.252:45501)
0807 22:55:50.487873 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.487883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.488446 (+ 563us) server_negotiation.cc:407] Connection header received
0807 22:55:50.488777 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.488785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.488886 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.489016 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.489868 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40218: BlockingRecv error: recv got EOF from 127.0.0.1:40218 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:55:50.501317 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.495289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.9.194.253:35425)
0807 22:55:50.495538 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.495547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.496083 (+ 536us) server_negotiation.cc:407] Connection header received
0807 22:55:50.496289 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.496294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.496381 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.496475 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.496896 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.497878 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.499544 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.499982 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.500121 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.500506 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.500636 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.500733 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.500853 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.500965 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 22:55:50.510051 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.507786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.9.194.252:45501)
0807 22:55:50.508044 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.508053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.508586 (+ 533us) server_negotiation.cc:407] Connection header received
0807 22:55:50.508790 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.508795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.508885 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.508989 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.509801 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40230: BlockingRecv error: recv got EOF from 127.0.0.1:40230 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:55:50.523190 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.517464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.9.194.253:35425)
0807 22:55:50.517768 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.517777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.518343 (+ 566us) server_negotiation.cc:407] Connection header received
0807 22:55:50.518562 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.518568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.518652 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.518757 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.519203 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.520076 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.521375 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.521846 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.521984 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.522410 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.522546 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.522646 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.522788 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.522869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
W20260807 22:55:50.536957 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.534513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.9.194.252:45501)
0807 22:55:50.534890 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.534900 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.535480 (+ 580us) server_negotiation.cc:407] Connection header received
0807 22:55:50.535722 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.535728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.535816 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.535926 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.536731 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40246: BlockingRecv error: recv got EOF from 127.0.0.1:40246 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
I20260807 22:55:50.554762 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.548195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.9.194.253:35425)
0807 22:55:50.548447 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.548456 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.548995 (+ 539us) server_negotiation.cc:407] Connection header received
0807 22:55:50.549243 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.549247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.549325 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.549431 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.549865 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.550768 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.552604 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.553125 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.553273 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.553786 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.553982 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.554058 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.554284 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.554369 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 22:55:50.563725 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.560722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.9.194.252:45501)
0807 22:55:50.560974 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.560987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.562170 (+ 1183us) server_negotiation.cc:407] Connection header received
0807 22:55:50.562395 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.562403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.562515 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.562637 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.563408 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40258: BlockingRecv error: recv got EOF from 127.0.0.1:40258 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:55:50.589210 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.583347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.9.194.253:35425)
0807 22:55:50.583618 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.583627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.584168 (+ 541us) server_negotiation.cc:407] Connection header received
0807 22:55:50.584413 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.584428 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.584498 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.584604 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.585039 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.585858 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.587215 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.587749 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.587893 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.588358 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.588498 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.588602 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.588760 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.588852 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260807 22:55:50.599031 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.595997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.9.194.252:45501)
0807 22:55:50.596256 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.596265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.597437 (+ 1172us) server_negotiation.cc:407] Connection header received
0807 22:55:50.597688 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.597694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.597781 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.597896 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.598778 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40268: BlockingRecv error: recv got EOF from 127.0.0.1:40268 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:55:50.642463 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.635025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.9.194.253:35425)
0807 22:55:50.635237 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.635243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.635849 (+ 606us) server_negotiation.cc:407] Connection header received
0807 22:55:50.636073 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.636080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.636204 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.636332 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.636801 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.638007 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.640010 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.640798 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.640941 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.641472 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.641612 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.641711 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.641876 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.641971 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 22:55:50.659922 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.657360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.9.194.252:45501)
0807 22:55:50.657588 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.657595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.658217 (+ 622us) server_negotiation.cc:407] Connection header received
0807 22:55:50.658447 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.658454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.658578 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.658704 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.659660 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40272: BlockingRecv error: recv got EOF from 127.0.0.1:40272 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:55:50.734560 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.728084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.9.194.253:35425)
0807 22:55:50.728322 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.728330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.728756 (+ 426us) server_negotiation.cc:407] Connection header received
0807 22:55:50.728957 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.728962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.729064 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.729202 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.729602 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.730811 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.732364 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.733069 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.733206 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.733674 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.733822 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.733953 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.734151 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.734252 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 22:55:50.743525 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.741179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.9.194.252:45501)
0807 22:55:50.741412 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.741418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.741874 (+ 456us) server_negotiation.cc:407] Connection header received
0807 22:55:50.742045 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.742051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.742151 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.742280 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.743253 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40280: BlockingRecv error: recv got EOF from 127.0.0.1:40280 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:55:50.883031 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:50.876379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.9.194.253:35425)
0807 22:55:50.876597 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.876606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.877003 (+ 397us) server_negotiation.cc:407] Connection header received
0807 22:55:50.877316 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.877323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.877414 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.877487 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:50.878233 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.879406 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.881406 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:50.881844 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:50.881947 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:50.882401 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:50.882516 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:50.882587 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:50.882742 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:55:50.882807 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 22:55:50.890935 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:50.888879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.9.194.252:45501)
0807 22:55:50.889097 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:50.889103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:50.889323 (+ 220us) server_negotiation.cc:407] Connection header received
0807 22:55:50.889553 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:50.889560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:50.889651 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:50.889760 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:50.890572 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40282: BlockingRecv error: recv got EOF from 127.0.0.1:40282 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 22:55:51.156998 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.150991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.9.194.253:35425)
0807 22:55:51.151246 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.151255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.151464 (+ 209us) server_negotiation.cc:407] Connection header received
0807 22:55:51.151701 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.151708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.151872 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.151995 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.152409 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.153774 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.155159 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.155691 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.155837 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.156262 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.156366 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.156476 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.156636 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.156719 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 22:55:51.166194 1876 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:51.163577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.9.194.252:45501)
0807 22:55:51.163806 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.163811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.164261 (+ 450us) server_negotiation.cc:407] Connection header received
0807 22:55:51.164603 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.164607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.164683 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.164770 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:51.165887 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40298: BlockingRecv error: recv got EOF from 127.0.0.1:40298 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 22:55:51.178714 1919 catalog_manager.cc:1259] Loaded cluster ID: 1d32dc3d06ef48739b27f6478da1cd7a
I20260807 22:55:51.178965 1919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: loading cluster ID for follower catalog manager: success
I20260807 22:55:51.182988 1919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: acquiring CA information for follower catalog manager: success
I20260807 22:55:51.185724 1919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e59d2bbae56349b784bbc4e52c2e3ef7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:55:51.688378 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.682943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.9.194.253:35425)
0807 22:55:51.683495 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.683504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.683587 (+ 83us) server_negotiation.cc:407] Connection header received
0807 22:55:51.683833 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.683838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.683906 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.684025 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.684495 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.685280 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.686622 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.687035 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.687176 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.687635 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.687740 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.687842 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.687991 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.688074 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":282,"threads_started":1}
I20260807 22:55:51.700350 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.694302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.9.194.252:45501)
0807 22:55:51.694835 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.694842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.694954 (+ 112us) server_negotiation.cc:407] Connection header received
0807 22:55:51.695159 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.695165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.695324 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.695482 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.695876 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.696560 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.697915 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.698416 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.698555 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.699094 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.699734 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.699902 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.699975 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.700062 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":257,"threads_started":1}
I20260807 22:55:51.717079 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.706931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.9.194.253:35425)
0807 22:55:51.707204 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.707214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.710540 (+ 3326us) server_negotiation.cc:407] Connection header received
0807 22:55:51.710744 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.710748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.710868 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.710997 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.712182 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.713101 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.714853 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.715486 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.715613 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.716145 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.716287 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.716383 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.716676 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.716774 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:55:51.718024 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.707694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.9.194.252:45501)
0807 22:55:51.707906 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.707915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.710823 (+ 2908us) server_negotiation.cc:407] Connection header received
0807 22:55:51.711005 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.711010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.711108 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.711209 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.711849 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.712797 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.715011 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.715693 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.715817 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.716263 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.716398 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.716493 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.717660 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.717738 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:55:51.718112 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:51.706851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.9.194.254:46185)
0807 22:55:51.711217 (+ 4366us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:51.711223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:51.711240 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:51.711362 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:51.711367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:51.711640 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:51.711882 (+ 242us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:51.712518 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.713508 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.715717 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:51.716283 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:51.716410 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:51.716954 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:51.717447 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:51.717578 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:51.717686 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 22:55:51.717769 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4232,"thread_start_us":4261,"threads_started":1}
I20260807 22:55:51.723340 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1635
I20260807 22:55:51.724581 1702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:51.838830 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1635
I20260807 22:55:51.863931 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1715
I20260807 22:55:51.865170 1782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:51.976402 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1715
I20260807 22:55:52.000178 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 1796
I20260807 22:55:52.001355 1867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:52.110301 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 1796
2026-08-07T22:55:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5200 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:55:52 dist-test-slave-5k54 krb5kdc[2093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:55:52 dist-test-slave-5k54 krb5kdc[2093](info): set up 2 sockets
Aug 07 22:55:52 dist-test-slave-5k54 krb5kdc[2093](info): commencing operation
krb5kdc: starting...
W20260807 22:55:53.306201 9995 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/test-user.keytab.
Aug 07 22:55:53 dist-test-slave-5k54 krb5kdc[2093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:55:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.194.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
Entry for principal kudu/127.9.194.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab.
I20260807 22:55:53.636798 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:34343
--webserver_interface=127.9.194.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:55:53.986582 2107 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:53.986981 2107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:53.987118 2107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:53.987252 2107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:53.994735 2107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:53.994835 2107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:53.994939 2107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:53.995029 2107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:53.995118 2107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:54.005376 2107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.254.keytab
--principal=kudu/127.9.194.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.254:34343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.194.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:54.006757 2107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:54.008189 2107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:54.020048 2114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:54.019999 2115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:54.020088 2117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:54.021275 2107 server_base.cc:1034] running on GCE node
I20260807 22:55:54.022006 2107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:54.023293 2107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:54.024521 2107 hybrid_clock.cc:648] HybridClock initialized: now 1786143354024457 us; error 34 us; skew 500 ppm
I20260807 22:55:54.024859 2107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:54.028265 2107 init.cc:377] Logged in from keytab as kudu/127.9.194.254@KRBTEST.COM (short username kudu)
I20260807 22:55:54.029690 2107 webserver.cc:460] Webserver started at http://127.9.194.254:40001/ using document root <none> and password file <none>
I20260807 22:55:54.030385 2107 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:54.030504 2107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:54.030849 2107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:54.034086 2107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data/instance:
uuid: "1c05fdc6a3e4453d925ea749a87f1b09"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "fb322822973fcacf517bba6ce98f455f"
server_key_iv: "9fc96cee778f541fa013ec1f09a8b9d0"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.034965 2107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance:
uuid: "1c05fdc6a3e4453d925ea749a87f1b09"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "fb322822973fcacf517bba6ce98f455f"
server_key_iv: "9fc96cee778f541fa013ec1f09a8b9d0"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.040284 2107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 22:55:54.043887 2124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:54.045286 2107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:55:54.045482 2107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal
uuid: "1c05fdc6a3e4453d925ea749a87f1b09"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "fb322822973fcacf517bba6ce98f455f"
server_key_iv: "9fc96cee778f541fa013ec1f09a8b9d0"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.045703 2107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:54.062815 2107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:54.066937 2107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:54.067206 2107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:54.086606 2107 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.254:34343
I20260807 22:55:54.086627 2176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.254:34343 every 8 connection(s)
I20260807 22:55:54.088070 2107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/data/info.pb
I20260807 22:55:54.088606 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 2107
I20260807 22:55:54.088840 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-0/wal/instance
I20260807 22:55:54.089458 9995 external_mini_cluster.cc:1428] Setting key d1180208bd15e0e57b519046c3a56f75
I20260807 22:55:54.092291 2177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:54.098922 2177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:54.122225 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.102215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.9.194.254:34343)
0807 22:55:54.104100 (+ 1885us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:54.104111 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:54.104179 (+ 68us) server_negotiation.cc:407] Connection header received
0807 22:55:54.105778 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:54.105787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:54.106100 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:54.106314 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:54.107703 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.108904 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.110283 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.110824 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.111024 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.114857 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:54.114865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:54.114885 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:54.114913 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:54.117733 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.118732 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.118739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.118741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.118928 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.119417 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.119456 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.119461 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.119997 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:54.120152 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:54.120907 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 22:55:54.121158 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1644,"spinlock_wait_cycles":31104,"thread_start_us":1404,"threads_started":1}
I20260807 22:55:54.122278 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.102215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:34343 (local address 127.0.0.1:37368)
0807 22:55:54.103263 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.103371 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:54.105418 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:54.106330 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:54.106337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:54.107078 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:54.107483 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.107511 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.109057 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.109064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.110131 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.110142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.111063 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.111070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.111242 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.112011 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:54.112033 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:54.114608 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:54.117882 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.117897 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.117912 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.118588 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.119054 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.119062 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.119069 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.119298 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.120163 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:54.120175 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:54.120451 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 22:55:54.120744 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 22:55:54.121372 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":74,"thread_start_us":437,"threads_started":1}
WARNING: no policy specified for kudu/127.9.194.253@KRBTEST.COM; defaulting to no policy
I20260807 22:55:54.125391 2177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.253@KRBTEST.COM" created.
I20260807 22:55:54.126505 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.126107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56578)
0807 22:55:54.126266 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.126400 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:55:54.126739 2127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
I20260807 22:55:54.128432 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.127979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56582)
0807 22:55:54.128153 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.128250 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:55:54.128643 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:38857: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
Entry for principal kudu/127.9.194.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab.
I20260807 22:55:54.163825 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:38857
--webserver_interface=127.9.194.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:55:54.177984 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } attempt: 1
I20260807 22:55:54.179116 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.178666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56586)
0807 22:55:54.178861 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.178986 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 22:55:54.180213 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.179833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56602)
0807 22:55:54.180005 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.180083 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 22:55:54.180414 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:38857: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
I20260807 22:55:54.248674 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } attempt: 2
I20260807 22:55:54.249923 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.249413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56616)
0807 22:55:54.249685 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.249777 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 22:55:54.251019 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.250634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:56632)
0807 22:55:54.250811 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.250897 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 22:55:54.251205 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:38857: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
I20260807 22:55:54.342450 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } attempt: 3
I20260807 22:55:54.344170 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.343319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44180)
0807 22:55:54.343732 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.343948 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260807 22:55:54.345309 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.344937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44186)
0807 22:55:54.345083 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.345161 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:55:54.345592 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:38857: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
I20260807 22:55:54.488842 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } attempt: 4
I20260807 22:55:54.490186 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.489635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44198)
0807 22:55:54.489959 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.490049 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260807 22:55:54.491253 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.490916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44202)
0807 22:55:54.491049 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.491131 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 22:55:54.491485 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.253:38857: Network error: Client connection negotiation failed: client connection to 127.9.194.253:38857: connect: Connection refused (error 111)
W20260807 22:55:54.507138 2187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:54.507576 2187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:54.507699 2187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:54.507844 2187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:54.515399 2187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:54.515564 2187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:54.515690 2187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:54.515764 2187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:54.515880 2187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:54.526504 2187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.253.keytab
--principal=kudu/127.9.194.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.253:38857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.194.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:54.527967 2187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:54.529340 2187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:54.541457 2198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:54.541422 2196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:54.541730 2194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:54.542394 2187 server_base.cc:1034] running on GCE node
I20260807 22:55:54.543254 2187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:54.544639 2187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:54.545893 2187 hybrid_clock.cc:648] HybridClock initialized: now 1786143354545822 us; error 55 us; skew 500 ppm
I20260807 22:55:54.546221 2187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:54.549571 2187 init.cc:377] Logged in from keytab as kudu/127.9.194.253@KRBTEST.COM (short username kudu)
I20260807 22:55:54.550925 2187 webserver.cc:460] Webserver started at http://127.9.194.253:46001/ using document root <none> and password file <none>
I20260807 22:55:54.551663 2187 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:54.551811 2187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:54.552153 2187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:54.555101 2187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data/instance:
uuid: "963b2f54dc604e4788a001a5484ea984"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "9bf4169ce864978e8e534aa46ab312a5"
server_key_iv: "1cad1a6f9a5a7984e039e6c7e02d47b6"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.555912 2187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal/instance:
uuid: "963b2f54dc604e4788a001a5484ea984"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "9bf4169ce864978e8e534aa46ab312a5"
server_key_iv: "1cad1a6f9a5a7984e039e6c7e02d47b6"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.560964 2187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:55:54.564342 2205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:54.565809 2187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:55:54.566012 2187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal
uuid: "963b2f54dc604e4788a001a5484ea984"
format_stamp: "Formatted at 2026-08-07 22:55:54 on dist-test-slave-5k54"
server_key: "9bf4169ce864978e8e534aa46ab312a5"
server_key_iv: "1cad1a6f9a5a7984e039e6c7e02d47b6"
server_key_version: "encryptionkey@0"
I20260807 22:55:54.566241 2187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:54.601310 2187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:54.605963 2187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:54.606280 2187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:54.625311 2187 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.253:38857
I20260807 22:55:54.625334 2257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.253:38857 every 8 connection(s)
I20260807 22:55:54.626860 2187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/data/info.pb
I20260807 22:55:54.630548 2258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:54.637091 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 2187
I20260807 22:55:54.637305 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-1/wal/instance
I20260807 22:55:54.636955 2258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:54.637811 9995 external_mini_cluster.cc:1428] Setting key b1de3cb6c24ebda4a479608e4099388f
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:54.658034 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.639690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.9.194.254:34343)
0807 22:55:54.640229 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:54.640234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:54.641194 (+ 960us) server_negotiation.cc:407] Connection header received
0807 22:55:54.643080 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:54.643086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:54.643177 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:54.643297 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:54.644891 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.645671 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.647168 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.647674 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.647808 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.651878 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:54.651883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:54.651887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:54.651925 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:54.654837 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.655676 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.655681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.655683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.655804 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.656260 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.656269 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.656271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.656565 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:54.656726 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:54.657424 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 22:55:54.657651 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":310,"threads_started":1}
I20260807 22:55:54.659034 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.640265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:34343 (local address 127.0.0.1:47592)
0807 22:55:54.641023 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.641068 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:54.642810 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:54.643561 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:54.643568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:54.644110 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:54.644659 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.644675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.645841 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.645845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.647024 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.647036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.647905 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.647912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.648139 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.648835 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:54.648858 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:54.651640 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:54.655001 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.655009 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.655026 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.655546 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.655925 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.655938 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.655942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.656134 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.656706 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:54.656714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:54.657061 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 22:55:54.657317 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 22:55:54.657853 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":337,"threads_started":1}
I20260807 22:55:54.660840 2258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.194.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.194.252@KRBTEST.COM" created.
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:54.680400 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.661871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.9.194.253:38857)
0807 22:55:54.662564 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:54.662571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:54.662596 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:55:54.662637 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:54.662643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:54.662810 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:54.662925 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:54.663986 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.665082 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.667006 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.667630 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.667726 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.674404 (+ 6678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:54.674416 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:54.674430 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:54.674460 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:54.677398 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.678224 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.678230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.678232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.678349 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.678769 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.678773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.678774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.679142 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:54.679302 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:54.679786 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 22:55:54.680087 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":542,"thread_start_us":387,"threads_started":1}
I20260807 22:55:54.680485 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.661642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44204)
0807 22:55:54.661882 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.661906 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:54.662030 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:54.662987 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:54.662994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:54.663580 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:54.663793 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.663808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.665230 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.665239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.666835 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.666854 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.667816 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.667825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.668009 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.668632 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:54.668656 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:54.674176 (+ 5520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:54.677545 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.677553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.677559 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.678095 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.678458 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.678465 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.678471 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.678656 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.679273 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:54.679280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:54.679492 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 22:55:54.679645 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 22:55:54.679988 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 22:55:54.682973 2258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:54.684198 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.683621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51104)
0807 22:55:54.683829 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.683994 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:55:54.684475 2206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.686116 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.685694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51116)
0807 22:55:54.685861 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.685970 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 22:55:54.686486 2258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
Entry for principal kudu/127.9.194.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab.
I20260807 22:55:54.713266 9995 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
/tmp/dist-test-taskIi3wnX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:44861
--webserver_interface=127.9.194.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.194.212:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:55:54.735827 2258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 1
I20260807 22:55:54.737115 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.736604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51132)
0807 22:55:54.736817 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.736957 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 22:55:54.738106 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.737758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51136)
0807 22:55:54.737933 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.738006 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:55:54.738319 2258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.790792 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } attempt: 5
Aug 07 22:55:54 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:54.805936 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.791946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.9.194.253:38857)
0807 22:55:54.792119 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:54.792124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:54.792144 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:55:54.792296 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:54.792302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:54.792368 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:54.792465 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:54.793456 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.794192 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.795967 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.796418 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.796526 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.800354 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:54.800359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:54.800362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:54.800396 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:54.803201 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.803944 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.803950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.803953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.804085 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:54.804477 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:54.804482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:54.804492 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:54.804779 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:54.804958 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:54.805355 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 22:55:54.805582 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:55:54.805934 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.791673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44214)
0807 22:55:54.791988 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.792017 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:54.792153 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:54.792532 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:54.792538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:54.793068 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:54.793299 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.793311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.794335 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.794341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.795750 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:54.795768 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:54.796583 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:54.796589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:54.796887 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:54.797609 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:54.797633 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:54.800142 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:54.803355 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.803361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.803365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.803821 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.804200 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:54.804205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:54.804209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:54.804358 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:54.804920 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:54.804926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:54.805092 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 22:55:54.805232 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:55:54.805570 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 22:55:54.806532 2258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 2
I20260807 22:55:54.807184 2177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:54.807575 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.807076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51152)
0807 22:55:54.807275 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.807377 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 22:55:54.808357 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.807842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51166)
0807 22:55:54.808122 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.808189 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260807 22:55:54.808635 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.808235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51168)
0807 22:55:54.808418 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.808501 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 22:55:54.809191 2258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.809363 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.808952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51172)
0807 22:55:54.809082 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.809207 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 22:55:54.809571 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.860780 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 1
I20260807 22:55:54.862142 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.861586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51174)
0807 22:55:54.861822 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.861959 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 22:55:54.863169 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.862831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51190)
0807 22:55:54.862984 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.863068 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 22:55:54.863387 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.900441 2258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 3
I20260807 22:55:54.901724 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.901239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51194)
0807 22:55:54.901466 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.901561 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 22:55:54.902802 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.902373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51196)
0807 22:55:54.902620 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.902695 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260807 22:55:54.903023 2258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:54.931639 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 2
I20260807 22:55:54.932837 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.932335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51206)
0807 22:55:54.932585 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.932678 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 22:55:54.933883 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:54.933556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51212)
0807 22:55:54.933707 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:54.933771 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 22:55:54.934080 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:55.040287 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 3
I20260807 22:55:55.041512 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.041020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51220)
0807 22:55:55.041232 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.041328 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 22:55:55.042616 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.042236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51222)
0807 22:55:55.042442 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.042507 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 22:55:55.042827 2177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
I20260807 22:55:55.046208 2258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 4
I20260807 22:55:55.047185 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.046754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51238)
0807 22:55:55.046973 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.047049 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 22:55:55.048125 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.047801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51246)
0807 22:55:55.047961 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.048023 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:55:55.048322 2258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.194.252:44861: Network error: Client connection negotiation failed: client connection to 127.9.194.252:44861: connect: Connection refused (error 111)
W20260807 22:55:55.065315 2268 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:55:55.065689 2268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:55:55.065819 2268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:55:55.065968 2268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:55:55.073403 2268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:55:55.073526 2268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:55:55.073657 2268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:55:55.073746 2268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:55:55.073858 2268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:55:55.083907 2268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.194.212:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.194.254:34343,127.9.194.253:38857,127.9.194.252:44861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/krb5kdc/kudu_127.9.194.252.keytab
--principal=kudu/127.9.194.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.194.252:44861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.194.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:55:55.085295 2268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:55:55.086757 2268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:55:55.100142 2273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:55.100158 2274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:55:55.100186 2276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:55:55.101082 2268 server_base.cc:1034] running on GCE node
I20260807 22:55:55.102021 2268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:55:55.103516 2268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:55:55.104797 2268 hybrid_clock.cc:648] HybridClock initialized: now 1786143355104700 us; error 55 us; skew 500 ppm
I20260807 22:55:55.105160 2268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:55:55.112466 2268 init.cc:377] Logged in from keytab as kudu/127.9.194.252@KRBTEST.COM (short username kudu)
I20260807 22:55:55.114145 2268 webserver.cc:460] Webserver started at http://127.9.194.252:38437/ using document root <none> and password file <none>
I20260807 22:55:55.114945 2268 fs_manager.cc:359] Metadata directory not provided
I20260807 22:55:55.115096 2268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:55:55.115566 2268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:55:55.119093 2268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data/instance:
uuid: "06d6ab08165e4f42bb963edf5df82a49"
format_stamp: "Formatted at 2026-08-07 22:55:55 on dist-test-slave-5k54"
server_key: "13cbff3955df2464eadadfa5b1c555d2"
server_key_iv: "7c522eef5f26b870a7b2cc0a735e9353"
server_key_version: "encryptionkey@0"
I20260807 22:55:55.119988 2268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal/instance:
uuid: "06d6ab08165e4f42bb963edf5df82a49"
format_stamp: "Formatted at 2026-08-07 22:55:55 on dist-test-slave-5k54"
server_key: "13cbff3955df2464eadadfa5b1c555d2"
server_key_iv: "7c522eef5f26b870a7b2cc0a735e9353"
server_key_version: "encryptionkey@0"
I20260807 22:55:55.125702 2268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:55:55.129410 2283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:55:55.130771 2268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 22:55:55.130925 2268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data,/tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal
uuid: "06d6ab08165e4f42bb963edf5df82a49"
format_stamp: "Formatted at 2026-08-07 22:55:55 on dist-test-slave-5k54"
server_key: "13cbff3955df2464eadadfa5b1c555d2"
server_key_iv: "7c522eef5f26b870a7b2cc0a735e9353"
server_key_version: "encryptionkey@0"
I20260807 22:55:55.131131 2268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:55:55.144325 2268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:55:55.148216 2268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:55:55.148440 2268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:55:55.167150 2268 rpc_server.cc:307] RPC server started. Bound to: 127.9.194.252:44861
I20260807 22:55:55.167158 2336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.194.252:44861 every 8 connection(s)
I20260807 22:55:55.168610 2268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/data/info.pb
I20260807 22:55:55.172549 2337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:55:55.173420 9995 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu as pid 2268
I20260807 22:55:55.173606 9995 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIi3wnX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142983724342-9995-0/minicluster-data/master-2/wal/instance
I20260807 22:55:55.174075 9995 external_mini_cluster.cc:1428] Setting key 39e1d5137ff50e4ec0f0f58f9bef7ff8
I20260807 22:55:55.182546 2337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:55.199138 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.176047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.9.194.252:44861)
0807 22:55:55.176899 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.176913 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.176978 (+ 65us) server_negotiation.cc:407] Connection header received
0807 22:55:55.179343 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.179374 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.179962 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.180439 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.181927 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.183952 (+ 2025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.185499 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.186977 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.187241 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.191903 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.191916 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.191929 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.191971 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.195081 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.195849 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.195855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.195861 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.196010 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.196406 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.196410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.196412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.196891 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.197211 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.197730 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.198108 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":398,"threads_started":1}
I20260807 22:55:55.200927 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.185675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47600 (local address 127.9.194.254:34343)
0807 22:55:55.187071 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.187076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.187090 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:55:55.187146 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.187150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.187208 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.187338 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.188646 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.189376 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.191094 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.191560 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.191684 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.195221 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.195226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.195229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.195260 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.198101 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.198809 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.198815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.198826 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.198954 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.199390 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.199396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.199410 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.199729 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.199895 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.200315 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.200521 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":1180,"threads_started":1}
I20260807 22:55:55.200953 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.185633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.254:34343 (local address 127.0.0.1:47600)
0807 22:55:55.186529 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.186570 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:55.186708 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:55.187395 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:55.187402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:55.188235 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:55.188489 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.188497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.189523 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.189531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.190938 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.190949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.191787 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.191794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.191964 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.192614 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:55.192640 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:55.195024 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:55.198231 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.198241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.198248 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.198680 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.199066 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.199071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.199075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.199278 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.199862 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:55.199870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:55.200044 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 22:55:55.200211 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 22:55:55.200519 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":421,"threads_started":1}
I20260807 22:55:55.202430 2337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.254@KRBTEST.COM
I20260807 22:55:55.217442 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.203166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.253:38857 (local address 127.0.0.1:44216)
0807 22:55:55.203350 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.203369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:55.203509 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:55.203811 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:55.203816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:55.204267 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:55.204463 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.204471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.205806 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.205813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.207233 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.207248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.208215 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.208221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.208441 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.209141 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:55.209166 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:55.211705 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:55.214816 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.214821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.214825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.215336 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.215759 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.215765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.215770 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.215974 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.216516 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:55.216524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:55.216688 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 22:55:55.216852 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 22:55:55.217145 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:55:55.217559 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.203173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.9.194.253:38857)
0807 22:55:55.203362 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.203375 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.203466 (+ 91us) server_negotiation.cc:407] Connection header received
0807 22:55:55.203623 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.203627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.203683 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.203759 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.204622 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.205662 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.207394 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.208065 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.208192 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.211932 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.211937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.211941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.211974 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.214690 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.215513 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.215516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.215519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.215644 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.216097 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.216102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.216106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.216412 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.216524 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.216949 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.217178 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:55:55.218609 2337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:55:55.219364 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.202360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.9.194.254:34343)
0807 22:55:55.202566 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.202572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.202588 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:55:55.202647 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.202650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.202711 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.202812 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.204419 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.205574 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.207394 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.208059 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.208192 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.213034 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.213039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.213042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.213074 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.216299 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.217087 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.217094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.217100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.217245 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.217821 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.217826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.217830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.218198 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.218496 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.218802 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.219030 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:55:55.220031 2177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 4
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.252@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.254@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.194.253@KRBTEST.COM
I20260807 22:55:55.236263 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.220575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51280)
0807 22:55:55.220738 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.220756 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:55.220852 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:55.222040 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:55.222048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:55.222470 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:55.222607 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.222616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.224036 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.224042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.225353 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.225366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.226100 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.226106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.226311 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.226880 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:55.226909 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:55.229428 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:55.233560 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.233569 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.233574 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.234106 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.234496 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.234502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.234506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.234689 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.235204 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:55.235209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:55.235347 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 22:55:55.235529 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 22:55:55.235845 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 22:55:55.236366 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.219297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.9.194.252:44861)
0807 22:55:55.219597 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.219602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.219619 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:55.219672 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.219677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.219747 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.219858 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.220767 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.221838 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.223711 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.224299 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.224438 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.228412 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.228419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.228424 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.228461 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.232530 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.233708 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.233714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.233717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.233862 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.234384 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.234389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.234392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.235061 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.235221 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.235748 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.236013 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":187}
I20260807 22:55:55.236670 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.219273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51268)
0807 22:55:55.219469 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.219489 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:55.219588 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:55.219881 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:55.219886 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:55.220426 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:55.220627 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.220637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.222000 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.222007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.223588 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.223599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.224460 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.224468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.224712 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.225362 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:55.225392 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:55.228193 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:55.232686 (+ 4493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.232693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.232699 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.233340 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.233996 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.234001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.234008 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.234188 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.235272 (+ 1084us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:55.235277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:55.235480 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 22:55:55.235624 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:55:55.236223 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":184}
I20260807 22:55:55.237603 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.220927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.9.194.252:44861)
0807 22:55:55.221727 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.221731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.221752 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:55:55.221797 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.221803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.221867 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.222012 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.222765 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.223886 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.225495 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.225951 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.226085 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.229654 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.229659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.229666 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.229694 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.233396 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.234220 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.234228 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.234231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.234375 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.234787 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.234794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.234797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.235067 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.235218 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.237000 (+ 1782us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.237185 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":460,"threads_started":1}
I20260807 22:55:55.239352 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.221148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.9.194.253:38857)
0807 22:55:55.221599 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.221604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.221622 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:55:55.221670 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.221675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.221745 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.221846 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.223407 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.224341 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.226172 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.226847 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.226996 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.232120 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.232125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.232128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.232159 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.235349 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.236424 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.236430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.236434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.236598 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.237252 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.237257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.237260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.237531 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.237828 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.238700 (+ 872us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.238966 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329}
I20260807 22:55:55.242643 2337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: Bootstrap starting.
I20260807 22:55:55.244155 9995 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:55:55.246090 2337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:55.247176 2337 log.cc:824] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:55.246434 2151 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1c05fdc6a3e4453d925ea749a87f1b09"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47614
I20260807 22:55:55.250375 2337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: No bootstrap required, opened a new log
I20260807 22:55:55.252651 2177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: Bootstrap starting.
I20260807 22:55:55.256513 2177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:55.257529 2337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.257756 2177 log.cc:824] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:55.257889 2337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d6ab08165e4f42bb963edf5df82a49, State: Initialized, Role: FOLLOWER
I20260807 22:55:55.258388 2337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.259398 2346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.259820 2346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.260529 2337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:55.261067 2177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: No bootstrap required, opened a new log
I20260807 22:55:55.268715 2177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.269282 2177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c05fdc6a3e4453d925ea749a87f1b09, State: Initialized, Role: FOLLOWER
I20260807 22:55:55.269832 2177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.271224 2358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.271685 2358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.272658 2177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:55.272758 2151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:55:55.272974 2151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:55.276481 2151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.278724 2151 leader_election.cc:288] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [CANDIDATE]: Term 1 election: Requested vote from peers 963b2f54dc604e4788a001a5484ea984 (127.9.194.253:38857), 06d6ab08165e4f42bb963edf5df82a49 (127.9.194.252:44861)
I20260807 22:55:55.279652 2310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "06d6ab08165e4f42bb963edf5df82a49"
I20260807 22:55:55.280145 2310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:55.283728 2310 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c05fdc6a3e4453d925ea749a87f1b09 in term 1.
I20260807 22:55:55.284416 2125 leader_election.cc:302] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d6ab08165e4f42bb963edf5df82a49, 1c05fdc6a3e4453d925ea749a87f1b09; no voters:
I20260807 22:55:55.273643 2268 master_runner.cc:186] Error getting master registration for 127.9.194.254:34343: OK, instance_id { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" instance_seqno: 1786143354081180 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:55:55.284804 2268 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:55:55.284965 2358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 1 FOLLOWER]: Leader election won for term 1
W20260807 22:55:55.295264 2360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:55:55.295461 2360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:55:55.303758 2358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [term 1 LEADER]: Becoming Leader. State: Replica: 1c05fdc6a3e4453d925ea749a87f1b09, State: Running, Role: LEADER
I20260807 22:55:55.304636 2358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.316545 2365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c05fdc6a3e4453d925ea749a87f1b09. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.317061 2365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: This master's current role is: LEADER
W20260807 22:55:55.320116 2130 tablet.cc:2369] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:55.330391 2382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:55:55.344027 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.315716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.9.194.252:44861)
0807 22:55:55.315910 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.315917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.324602 (+ 8685us) server_negotiation.cc:407] Connection header received
0807 22:55:55.324890 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.324897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.325032 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.325114 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.327736 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.328803 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.331303 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.331991 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.332120 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.336643 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.336653 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.336658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.336706 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.340760 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.341577 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.341584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.341587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.341756 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.342201 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.342207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.342209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.342579 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.342797 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.343355 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.343625 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:55:55.344412 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.313596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.9.194.253:38857)
0807 22:55:55.313830 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.313837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.316875 (+ 3038us) server_negotiation.cc:407] Connection header received
0807 22:55:55.317267 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.317274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.317386 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.317473 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.322411 (+ 4938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.323631 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.327317 (+ 3686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.328287 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.328442 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.335186 (+ 6744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.335193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.335198 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.335248 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.339029 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.340883 (+ 1854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.340889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.340892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.341086 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.341982 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.341987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.341991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.342368 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.342620 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.343780 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.344037 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:55:55.347823 2258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } attempt: 5
I20260807 22:55:55.351095 2382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:55:55 dist-test-slave-5k54 krb5kdc[2093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786143354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.194.253@KRBTEST.COM for kudu/127.9.194.252@KRBTEST.COM
I20260807 22:55:55.359405 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.314178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.9.194.254:34343)
0807 22:55:55.314367 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.314373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.314722 (+ 349us) server_negotiation.cc:407] Connection header received
0807 22:55:55.319198 (+ 4476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.319206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.319312 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.319400 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.324581 (+ 5181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.325732 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.332830 (+ 7098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.333617 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.333729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.351876 (+ 18147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.351884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.351889 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.351933 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.355672 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.356728 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.356738 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.356742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.356928 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.357428 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.357434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.357438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.357818 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.358120 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.358738 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.358996 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:55:55.367408 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.348818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.194.252:44861 (local address 127.0.0.1:51300)
0807 22:55:55.349150 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:55:55.349195 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:55:55.349299 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:55:55.350291 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:55:55.350296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:55:55.350808 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:55:55.351031 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.351043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.352982 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.352987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.354775 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:55:55.354792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.355930 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.355938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:55:55.356198 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.356859 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:55:55.356881 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:55:55.359837 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:55:55.364425 (+ 4588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.364431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.364436 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.365018 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.365484 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:55:55.365489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:55:55.365493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:55:55.365679 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:55:55.366346 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:55:55.366351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:55:55.366516 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 22:55:55.366679 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 22:55:55.366993 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260807 22:55:55.369671 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:55.349814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.9.194.252:44861)
0807 22:55:55.349974 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.349981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.349998 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:55.350055 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.350059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.350127 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.351189 (+ 1062us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.351337 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.352492 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.355013 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:55.355744 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:55.360024 (+ 4280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:55.360296 (+ 272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:55:55.360302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:55:55.360306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:55:55.360354 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:55:55.364259 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.365188 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.365193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.365197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.365349 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:55:55.365822 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:55:55.365827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:55:55.365830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:55:55.366191 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:55:55.368806 (+ 2615us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:55.368972 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:55:55.369223 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:55:55.377687 2258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: Bootstrap starting.
I20260807 22:55:55.381919 2258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: Neither blocks nor log segments found. Creating new log.
I20260807 22:55:55.383283 2258 log.cc:824] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: Log is configured to *not* fsync() on all Append() calls
I20260807 22:55:55.387760 2310 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [term 1 FOLLOWER]: Refusing update from remote peer 1c05fdc6a3e4453d925ea749a87f1b09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:55.388810 2365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:55.375391 2107 master_runner.cc:186] Error getting master registration for 127.9.194.253:38857: OK, instance_id { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" instance_seqno: 1786143354620013 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:55:55.396128 2107 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 22:55:55.400594 2289 tablet.cc:2369] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:55:55.403964 2258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: No bootstrap required, opened a new log
I20260807 22:55:55.419827 2346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c05fdc6a3e4453d925ea749a87f1b09. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.420202 2346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.424141 2258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.424597 2258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 963b2f54dc604e4788a001a5484ea984, State: Initialized, Role: FOLLOWER
I20260807 22:55:55.425180 2258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } }
I20260807 22:55:55.432423 2393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.432878 2393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.434854 2258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:55:55.434942 2232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "963b2f54dc604e4788a001a5484ea984"
I20260807 22:55:55.435518 2232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:55:55.439302 2232 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c05fdc6a3e4453d925ea749a87f1b09 in term 1.
I20260807 22:55:55.440325 2231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [term 1 FOLLOWER]: Refusing update from remote peer 1c05fdc6a3e4453d925ea749a87f1b09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:55:55.441414 2358 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:55:55.451103 2358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.451444 2358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:55.455386 2382 catalog_manager.cc:1347] Generated new cluster ID: 51d28cc011fd45abaaf25fd27731b8b0
I20260807 22:55:55.455545 2382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:55:55.469027 2346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.469439 2346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.488548 2398 mvcc.cc:205] Tried to move back new op lower bound from 7316043183622262784 to 7316043183337472000. Current Snapshot: MvccSnapshot[applied={T|T < 7316043183622262784}]
I20260807 22:55:55.492679 2393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c05fdc6a3e4453d925ea749a87f1b09. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.493014 2393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.494292 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.494725 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:55:55.511790 2410 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:55:55.522837 2365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c05fdc6a3e4453d925ea749a87f1b09" member_type: VOTER last_known_addr { host: "127.9.194.254" port: 34343 } } peers { permanent_uuid: "963b2f54dc604e4788a001a5484ea984" member_type: VOTER last_known_addr { host: "127.9.194.253" port: 38857 } } peers { permanent_uuid: "06d6ab08165e4f42bb963edf5df82a49" member_type: VOTER last_known_addr { host: "127.9.194.252" port: 44861 } } }
I20260807 22:55:55.523150 2365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09 [sys.catalog]: This master's current role is: LEADER
I20260807 22:55:55.528715 2382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:55:55.530682 2382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:55:55.559350 2382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c05fdc6a3e4453d925ea749a87f1b09: Generated new TSK 0
I20260807 22:55:55.560621 2382 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:55:55.674156 2342 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.664770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.9.194.254:34343)
0807 22:55:55.665014 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.665022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.672126 (+ 7104us) server_negotiation.cc:407] Connection header received
0807 22:55:55.672453 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.672460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.672560 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.672643 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.673905 (+ 1262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47626: BlockingRecv error: recv got EOF from 127.0.0.1:47626 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 22:55:55.676573 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.664837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.9.194.253:38857)
0807 22:55:55.665023 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.665029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.674768 (+ 9739us) server_negotiation.cc:407] Connection header received
0807 22:55:55.675032 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.675038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.675132 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.675203 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.676341 (+ 1138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44238: BlockingRecv error: recv got EOF from 127.0.0.1:44238 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 22:55:55.679157 2343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.672415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.9.194.252:44861)
0807 22:55:55.672598 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.672605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.677507 (+ 4902us) server_negotiation.cc:407] Connection header received
0807 22:55:55.677751 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.677758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.677855 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.677929 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.678915 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51312: BlockingRecv error: recv got EOF from 127.0.0.1:51312 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 22:55:55.689539 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.684458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.9.194.253:38857)
0807 22:55:55.684638 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.684645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.687939 (+ 3294us) server_negotiation.cc:407] Connection header received
0807 22:55:55.688190 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.688197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.688318 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.688422 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.689267 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44244: BlockingRecv error: recv got EOF from 127.0.0.1:44244 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 22:55:55.701628 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.699467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.9.194.253:38857)
0807 22:55:55.699641 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.699650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.700154 (+ 504us) server_negotiation.cc:407] Connection header received
0807 22:55:55.700332 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.700339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.700434 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.700546 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.701412 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44258: BlockingRecv error: recv got EOF from 127.0.0.1:44258 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 22:55:55.708894 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.706782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.9.194.253:38857)
0807 22:55:55.706953 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.706958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.707493 (+ 535us) server_negotiation.cc:407] Connection header received
0807 22:55:55.707636 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.707642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.707745 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.707853 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.708653 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44266: BlockingRecv error: recv got EOF from 127.0.0.1:44266 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 22:55:55.718410 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.716380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.9.194.253:38857)
0807 22:55:55.716567 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.716572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.717046 (+ 474us) server_negotiation.cc:407] Connection header received
0807 22:55:55.717181 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.717186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.717289 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.717403 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.718172 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44282: BlockingRecv error: recv got EOF from 127.0.0.1:44282 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 22:55:55.736078 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.733627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.9.194.253:38857)
0807 22:55:55.733808 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.733826 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.734370 (+ 544us) server_negotiation.cc:407] Connection header received
0807 22:55:55.734598 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.734605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.734703 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.734822 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.735836 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44296: BlockingRecv error: recv got EOF from 127.0.0.1:44296 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 22:55:55.758996 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.756441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.9.194.253:38857)
0807 22:55:55.756648 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.756656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.757234 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:55:55.757467 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.757474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.757593 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.757726 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.758722 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44312: BlockingRecv error: recv got EOF from 127.0.0.1:44312 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 22:55:55.797609 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.795115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.9.194.253:38857)
0807 22:55:55.795324 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.795332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.795926 (+ 594us) server_negotiation.cc:407] Connection header received
0807 22:55:55.796126 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.796134 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.796261 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.796411 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.797288 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44322: BlockingRecv error: recv got EOF from 127.0.0.1:44322 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 22:55:55.867280 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.864992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.9.194.253:38857)
0807 22:55:55.865262 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:55.865269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:55.865698 (+ 429us) server_negotiation.cc:407] Connection header received
0807 22:55:55.865917 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:55.865921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:55.865995 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:55.866085 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:55.867024 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44326: BlockingRecv error: recv got EOF from 127.0.0.1:44326 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 22:55:56.002148 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:55.999773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.9.194.253:38857)
0807 22:55:56.000060 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.000067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.000421 (+ 354us) server_negotiation.cc:407] Connection header received
0807 22:55:56.000685 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.000689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.000768 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.000941 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:56.001835 (+ 894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44330: BlockingRecv error: recv got EOF from 127.0.0.1:44330 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260807 22:55:56.263679 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:55:56.261798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.9.194.253:38857)
0807 22:55:56.262037 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.262044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.262227 (+ 183us) server_negotiation.cc:407] Connection header received
0807 22:55:56.262455 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.262461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.262543 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.262654 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:55:56.263419 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44344: BlockingRecv error: recv got EOF from 127.0.0.1:44344 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:55:56.297381 2360 catalog_manager.cc:1259] Loaded cluster ID: 51d28cc011fd45abaaf25fd27731b8b0
I20260807 22:55:56.297485 2360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: loading cluster ID for follower catalog manager: success
I20260807 22:55:56.300168 2360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: acquiring CA information for follower catalog manager: success
I20260807 22:55:56.301872 2360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06d6ab08165e4f42bb963edf5df82a49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:55:56.516342 2410 catalog_manager.cc:1259] Loaded cluster ID: 51d28cc011fd45abaaf25fd27731b8b0
I20260807 22:55:56.516520 2410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: loading cluster ID for follower catalog manager: success
I20260807 22:55:56.519250 2410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: acquiring CA information for follower catalog manager: success
I20260807 22:55:56.521299 2410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 963b2f54dc604e4788a001a5484ea984: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:55:56.787287 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:56.779687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.9.194.253:38857)
0807 22:55:56.780656 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.780661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.780678 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:56.780747 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.780751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.780944 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.781057 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:56.781638 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.782540 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.784633 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.785153 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.785247 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:56.785916 (+ 669us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:56.786533 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:56.786644 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:56.786807 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 22:55:56.786903 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":684,"threads_started":1}
I20260807 22:55:56.800319 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:56.794288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.9.194.252:44861)
0807 22:55:56.794911 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.794916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.794938 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:55:56.795058 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.795062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.795213 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.795409 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:56.795826 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.796535 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.797930 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.798423 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.798560 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:56.799073 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:56.799673 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:56.799829 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:56.799929 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 22:55:56.800013 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":381,"threads_started":1}
I20260807 22:55:56.822721 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:56.806724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.9.194.253:38857)
0807 22:55:56.806923 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.806929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.812160 (+ 5231us) server_negotiation.cc:407] Connection header received
0807 22:55:56.812543 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.812553 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.812680 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.812768 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:56.813703 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.815336 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.819493 (+ 4157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.820271 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.820370 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:56.821704 (+ 1334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:56.821963 (+ 259us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:56.822047 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:56.822285 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 22:55:56.822379 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:55:56.826675 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:56.806971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.9.194.254:34343)
0807 22:55:56.818698 (+ 11727us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.818702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.818719 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:55:56.818829 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.818833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.819004 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.819226 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:56.819906 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.820995 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.823130 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.824094 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.824212 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:56.824995 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:56.825534 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:56.825642 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:56.826248 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 22:55:56.826357 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11606,"thread_start_us":11495,"threads_started":1}
I20260807 22:55:56.831259 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:55:56.807941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.9.194.252:44861)
0807 22:55:56.808086 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 22:55:56.808090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:55:56.824208 (+ 16118us) server_negotiation.cc:407] Connection header received
0807 22:55:56.824673 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:55:56.824678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:55:56.824831 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:55:56.824898 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:55:56.825893 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.827044 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.828837 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:55:56.829508 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:55:56.829646 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:55:56.830431 (+ 785us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:55:56.830578 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:55:56.830682 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:55:56.830856 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 22:55:56.830944 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 22:55:56.831964 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 2107
I20260807 22:55:56.833065 2171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:56.970778 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 2107
I20260807 22:55:56.996750 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 2187
I20260807 22:55:56.997828 2253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:57.103659 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 2187
I20260807 22:55:57.127588 9995 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu pid 2268
I20260807 22:55:57.128702 2332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:55:57.235906 9995 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIi3wnX/build/asan/bin/kudu with pid 2268
2026-08-07T22:55:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5120 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10320 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (373458 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST